[Mesa-dev] Enabling Mesa video frontends on top of D3D12 gallium driver

[Dave Airlie]

On Thu, 18 Nov 2021 at 18:45, Sil Vilerino
<Silvio.Vilerino at microsoft.com> wrote:
>
> Hello mesa-dev community,
>
>
>
> We are starting to work on adding support for D3D12 Video acceleration in the mesa gallium D3D12 driver so that mesa frontends such as VA, VDPAU, etc can leverage HW acceleration in those environments.
>
> To begin with we are working in a mesa fork project on a simple video decoder prototype that implements the pipe_video_codec and pipe_video_buffer interfaces with a D3D12 backend.
>

Welcome, to start I'm not authorative on this area of Mesa at all,
I've just started investigating vaapi and vulkan video decode.

I'm not really sure who understands all the ins/outs, I've cc'ed two
AMD contacts who have worked on this code.

> Wayland/Software screen creation in VA_DRIVER_INIT_FUNC
> In our d3d12 gallium driver, we rely on the EGL/GLX and the DRI frontend to handle the pure swrast screen creation as our virtualized environment doesn’t have devices listed under /dev/dri/*.
> The default for gstreamer/vainfo initialization code in WSL seems to be to create a Wayland display and pass it to VAInitialize. If we go ahead and create a pure software wayland d3d12_screen in VA_DRIVER_INIT_FUNC(VADriverContextP ctx) we hit vaGetSurfaceBufferWl() is not implemented at VAAPI Gallium state tracker (#587) · Issues · Mesa / mesa · GitLab when trying to run a simple gstreamer pipeline that decodes with VAAPI (and d3d12 video backend) and presents to screen in a display sink.
> From vaGetSurfaceBufferWl() is not implemented at VAAPI Gallium state tracker (#587) · Issues · Mesa / mesa · GitLab discussion, it looks like “the change removing "NOT_IMPLEMENTED" is wayland display should be opened with DRM code path, and it's already implemented. the code here is not general switch to turn on the wayland support on vaapi, it's just one of the steps to complete that support and which has been implemented.”
> Could you please provide some details on the high level design of how wayland supports for libva and gallium video drivers ?
> Which of the currently implemented paths in mesa is currently recommended in general for video driver implementors: Wayland with DRM fd device or X DRI/DR3?
> What’d be a recommended way of supporting pure swrast screens like d3d12 for libva in VA_DRIVER_INIT_FUNC?
>
> Are there any objections to having a pure sw (no DRM devices) screen ?
> Another alternative we discussed was to enable VGEM in the WSL kernel and create a wayland swrast screen using a kms_dri_create_winsys with the virtual DRM device FD but that’d still require more work to support wayland presentation to screen code paths (see next section below).
> Recommended present path
> Assuming we create a wayland screen (pure software or using VGEM + DRM) in VA_DRIVER_INIT_FUNC, once end_frame() finishes execution and we have the decoded image in pipe_video_buffer *target, what are the supported and recommended paths for presenting to screen?
> Looks like both VDPAU (vlVdpPresentationQueueDisplay) and VAAPI (vlVaPutSurface) call texture_from_drawable() to pass the associated pipe_resource texture to flush_frontbuffer(), but the texture_from_drawable() function seems only implemented on X backends (vl_winsys_dri3/vl_winsys_dri.c) and not for vl_winsys_swrast/vl_winsys_drm. As it’s mentioned that there’s support for wayland on vaapi, we’d like to get some clarity on how the presentation path is designed here.

My expectation here is that making it operate like the GL paths is
probably what you want, and that if you are not using wayland there,
you shouldn't be doing it here. If you are there then adding support
for it here would be required.

I haven't investigated vaapi on wayland at all, and I probably need to
look into that soon, I'd expect adding proper support for presenting
would be a good feature to add there.

Currently at least using things like mplayer -hwdec=vaapi, the vaapi
code is just used to do the decode into dma-buf on Linux and then that
is handed off to a separate GL context for presentation.

Dave.