[Mesa-dev] Enabling Mesa video frontends on top of D3D12 gallium driver

[Lucas Stach]

Am Montag, dem 22.11.2021 um 13:01 +0100 schrieb Christian König:
> Hi guys,
> 
> Am 22.11.21 um 06:49 schrieb Dave Airlie:
> > On Thu, 18 Nov 2021 at 18:45, Sil Vilerino
> > <Silvio.Vilerino at microsoft.com> wrote:
> > > Hello mesa-dev community,
> > > 
> > > 
> > > 
> > > We are starting to work on adding support for D3D12 Video acceleration in the mesa gallium D3D12 driver so that mesa frontends such as VA, VDPAU, etc can leverage HW acceleration in those environments.
> > > 
> > > To begin with we are working in a mesa fork project on a simple video decoder prototype that implements the pipe_video_codec and pipe_video_buffer interfaces with a D3D12 backend.
> > > 
> > Welcome, to start I'm not authorative on this area of Mesa at all,
> > I've just started investigating vaapi and vulkan video decode.
> > 
> > I'm not really sure who understands all the ins/outs, I've cc'ed two
> > AMD contacts who have worked on this code.
> 
> Yeah, I'm not working on that for a long time now but I think I still 
> have all the design pieces in my head.
> 
> > > Wayland/Software screen creation in VA_DRIVER_INIT_FUNC
> > > In our d3d12 gallium driver, we rely on the EGL/GLX and the DRI frontend to handle the pure swrast screen creation as our virtualized environment doesn’t have devices listed under /dev/dri/*.
> > > The default for gstreamer/vainfo initialization code in WSL seems to be to create a Wayland display and pass it to VAInitialize. If we go ahead and create a pure software wayland d3d12_screen in VA_DRIVER_INIT_FUNC(VADriverContextP ctx) we hit vaGetSurfaceBufferWl() is not implemented at VAAPI Gallium state tracker (#587) · Issues · Mesa / mesa · GitLab when trying to run a simple gstreamer pipeline that decodes with VAAPI (and d3d12 video backend) and presents to screen in a display sink.
> > >  From vaGetSurfaceBufferWl() is not implemented at VAAPI Gallium state tracker (#587) · Issues · Mesa / mesa · GitLab discussion, it looks like “the change removing "NOT_IMPLEMENTED" is wayland display should be opened with DRM code path, and it's already implemented. the code here is not general switch to turn on the wayland support on vaapi, it's just one of the steps to complete that support and which has been implemented.”
> > > Could you please provide some details on the high level design of how wayland supports for libva and gallium video drivers ?
> > > Which of the currently implemented paths in mesa is currently recommended in general for video driver implementors: Wayland with DRM fd device or X DRI/DR3?
> > > What’d be a recommended way of supporting pure swrast screens like d3d12 for libva in VA_DRIVER_INIT_FUNC?
> 
> Well long story short: That simply won't work that easily.
> 
> As far as I know interop with Wayland only works by passing DMA-buf 
> handles around. And the problem is now that software drivers can't 
> create any DMA-buf handles because they don't have an underlying kernel 
> driver.
> 
> This question already came up a couple of times now in different contexts.
> 
> > > Are there any objections to having a pure sw (no DRM devices) screen ?
> > > Another alternative we discussed was to enable VGEM in the WSL kernel and create a wayland swrast screen using a kms_dri_create_winsys with the virtual DRM device FD but that’d still require more work to support wayland presentation to screen code paths (see next section below).
> > > Recommended present path
> > > Assuming we create a wayland screen (pure software or using VGEM + DRM) in VA_DRIVER_INIT_FUNC, once end_frame() finishes execution and we have the decoded image in pipe_video_buffer *target, what are the supported and recommended paths for presenting to screen?
> > > Looks like both VDPAU (vlVdpPresentationQueueDisplay) and VAAPI (vlVaPutSurface) call texture_from_drawable() to pass the associated pipe_resource texture to flush_frontbuffer(), but the texture_from_drawable() function seems only implemented on X backends (vl_winsys_dri3/vl_winsys_dri.c) and not for vl_winsys_swrast/vl_winsys_drm. As it’s mentioned that there’s support for wayland on vaapi, we’d like to get some clarity on how the presentation path is designed here.
> > My expectation here is that making it operate like the GL paths is
> > probably what you want, and that if you are not using wayland there,
> > you shouldn't be doing it here. If you are there then adding support
> > for it here would be required.
> > 
> > I haven't investigated vaapi on wayland at all, and I probably need to
> > look into that soon, I'd expect adding proper support for presenting
> > would be a good feature to add there.
> > 
> > Currently at least using things like mplayer -hwdec=vaapi, the vaapi
> > code is just used to do the decode into dma-buf on Linux and then that
> > is handed off to a separate GL context for presentation.
> 
> Yeah, exactly that's the problem. The pure software driver somehow needs 
> to get a DMA-buf file descriptor.
> 
> In theory you can maybe use memfd() as long as you don't import the file 
> descriptor anyway. But I'm not sure what a memfd() file descriptor says 
> if it sees DMA-buf IOCTLs.
> 
Not sure how much it's worth for the issue at hand, but udmabuf
(drivers/dma-buf/udmabuf.c) allows to create a proper dmabuf from a
memfd handle.

Regards,
Lucas

> To summarize using VGEM for that sounds like the easiest to implement 
> approach to me as well.
> 
> Regards,
> Christian.
> 
> > 
> > Dave.
>