[Mesa-dev] Enabling Mesa video frontends on top of D3D12 gallium driver

Wed Nov 17 22:45:39 UTC 2021

Hello mesa-dev community,

We are starting to work on adding support for D3D12 Video acceleration in the mesa gallium D3D12 driver so that mesa frontends such as VA, VDPAU, etc can leverage HW acceleration in those environments.
To begin with we are working in a mesa fork project<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.freedesktop.org%2Fsivileri%2Fmesa%2F-%2Fcommits%2Fuser%2Fsivileri%2Fmesa_d3d12_video_driver_1%2F&data=04%7C01%7CSilvio.Vilerino%40microsoft.com%7C05685c59e1214a5452b008d9aa1afa74%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637727855121787525%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0SqJ10OJ8Pvvpk0z7v8T9GX1OS%2FfhCGCoa6RHuSElpk%3D&reserved=0> on a simple video decoder prototype that implements the pipe_video_codec and pipe_video_buffer interfaces with a D3D12 backend.

We have a couple of questions and would appreciate your input and feedback.

Wayland/Software screen creation in VA_DRIVER_INIT_FUNC

In our d3d12 gallium driver, we rely on the EGL/GLX and the DRI frontend to handle the pure swrast screen creation as our virtualized environment doesn't have devices listed under /dev/dri/*.

The default for gstreamer/vainfo initialization code in WSL seems to be to create a Wayland display and pass it to VAInitialize. If we go ahead and create a pure software wayland d3d12_screen in VA_DRIVER_INIT_FUNC(VADriverContextP ctx) we hit vaGetSurfaceBufferWl() is not implemented at VAAPI Gallium state tracker (#587) * Issues * Mesa / mesa * GitLab<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.freedesktop.org%2Fmesa%2Fmesa%2F-%2Fissues%2F587&data=04%7C01%7CSilvio.Vilerino%40microsoft.com%7C05685c59e1214a5452b008d9aa1afa74%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637727855121797520%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IYMZ3LSXUy%2BhMkmkh1DW%2FdkJ7M2GOu29CJFAESeQWCg%3D&reserved=0> when trying to run a simple gstreamer pipeline that decodes with VAAPI (and d3d12 video backend) and presents to screen in a display sink.

>From vaGetSurfaceBufferWl() is not implemented at VAAPI Gallium state tracker (#587) * Issues * Mesa / mesa * GitLab<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.freedesktop.org%2Fmesa%2Fmesa%2F-%2Fissues%2F587&data=04%7C01%7CSilvio.Vilerino%40microsoft.com%7C05685c59e1214a5452b008d9aa1afa74%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637727855121797520%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IYMZ3LSXUy%2BhMkmkh1DW%2FdkJ7M2GOu29CJFAESeQWCg%3D&reserved=0> discussion, it looks like "the change removing<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.freedesktop.org%2Fmesa%2Fmesa%2F-%2Fcommit%2F2f90d11d86e476c679abbc0420ea22bd14b53ef7&data=04%7C01%7CSilvio.Vilerino%40microsoft.com%7C05685c59e1214a5452b008d9aa1afa74%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637727855121807515%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9CkVhU5eCy7bvPx6nmmA43Fw9Vev%2BB5U2O5V7wuL9sU%3D&reserved=0> "NOT_IMPLEMENTED" is wayland display should be opened with DRM code path, and it's already implemented. the code here is not general switch to turn on the wayland support on vaapi, it's just one of the steps to complete that support and which has been implemented."

  1.  Could you please provide some details on the high level design of how wayland supports for libva and gallium video drivers ?
  2.  Which of the currently implemented paths in mesa is currently recommended in general for video driver implementors: Wayland with DRM fd device or X DRI/DR3?
  3.  What'd be a recommended way of supporting pure swrast screens like d3d12 for libva in VA_DRIVER_INIT_FUNC?
     *   Are there any objections to having a pure sw (no DRM devices) screen ?
     *   Another alternative we discussed was to enable VGEM in the WSL kernel and create a wayland swrast screen using a kms_dri_create_winsys with the virtual DRM device FD but that'd still require more work to support wayland presentation to screen code paths (see next section below).

Recommended present path

Assuming we create a wayland screen (pure software or using VGEM + DRM) in VA_DRIVER_INIT_FUNC, once end_frame() finishes execution and we have the decoded image in pipe_video_buffer *target, what are the supported and recommended paths for presenting to screen?

Looks like both VDPAU (vlVdpPresentationQueueDisplay) and VAAPI (vlVaPutSurface) call texture_from_drawable() to pass the associated pipe_resource texture to flush_frontbuffer(), but the texture_from_drawable() function seems only implemented on X backends (vl_winsys_dri3/vl_winsys_dri.c) and not for vl_winsys_swrast/vl_winsys_drm. As it's mentioned that there's support for wayland on vaapi, we'd like to get some clarity on how the presentation path is designed here.

Thanks,
Sil Vilerino

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20211117/b14a344f/attachment-0001.htm>