[PATCH 00/10] dcc: Create a stream for non-gl/remote clients that want to use dmabuf (v9)

[Michael Scherle]

Oh I made a mistake and send this to the wrong mailing list, I'm
so sorry, I wanted to send it to the spice devel.


On 27.02.25 12:03, Michael Scherle wrote:
> This merge request is based on Vivek Kasireddy's
> [PATCH v8 0/6] dcc: Create a stream for non-gl/remote clients that
> want to use dmabuf (v8), who gave me the permission to pursue the
> merge request. I also submitted this directly on gitlab but but I'm
> now also submitting it via the mailing list .I have made my changes
> in separate commits, should I merge them with the existing commits?
> 
> Original Description:
> For clients that cannot accept a dmabuf fd directly (such as those
> running on a remote system), this patch series provides a way for
> the Spice server to stream the gl/dmabuf data/buffer instead. This
> is mostly done by enabling the creation of Gst memory using a dmabuf
> fd as the source. This ability is useful given that dmabuf is the
> standard mechanism for sharing buffers between various drivers and
> userspace in many Graphics and Media usecases. Currently, this is
> tested with Qemu and remote-viewer using the x264enc/avdec_h264
> and msdkh264enc/dec plugins to stream the Guest/VM desktop but it
> can be easily extended to other plugins and applications.
> Here is roughly how things work:
> 
> The application (e.g, Qemu) chooses its preferred codec (a Gstreamer
> one) and calls gl_scanout (to update the fd) followed by gl_draw.
> In response, the Spice server checks to see if the client is capable
> of accepting a dmabuf fd directly or not. If yes, the fd is forwarded
> directly to the client; otherwise, a new stream is created.
> The Spice server then sends the dmabuf fd to the Gstreamer encoder
> which uses it as an input for creating an encoded buffer which is then
> sent to the client.
> Once the encoding process is done, an async completion cookie is sent
> to the application.
> 
> Here is a link to the previous version that used a drawable to share
> the dmabuf fd with the Gstreamer encoder:
> https://lists.freedesktop.org/archives/spice-devel/2023-January/052948.html
> This version is tested together with following (required) patches in qemu:
> https://gitlab.freedesktop.org/Vivek/qemu/-/commits/spice_gl_on_v4
> 
> Changelog:
> v9:
> 
> updated the spice-common submodule, which is needed for udev helper to identify GPU vendor
> include dmabuf encoding only if dmabuf supported and gstreamer >= 1.24
> fix a race condition when the video stream is stopped while encoding a frame
> 
> v8:
> 
> Added a new gstreamer-encoder patch to this series to convert drm format
> shared by the VMM to the appropriate Gstreamer format.
> 
> v7:
> 
> Revert back to the previous design where we do not share fd with the stream
> and scanout is the sole owner of the fd. This is because sharing fd ownership
> opens up a lot of corner cases.
> 
> v6: (Frediano)
> 
> Properly share ownership of the dmabuf fd between stream and scanout
> Ensure that a newly created stream is associated with all connected clients
> 
> v5:
> 
> Addressed review comments from Frediano mainly regarding adding autoconf
> support for gstreamer-allocators dependency and not needing to access
> scanout as part of gl draw operation
> 
> v4:
> 
> Test with Virgl enabled
> Associate dmabuf's y0_top with stream's top_down variable
> 
> v3:
> 
> Updated the second patch to have a new primary surface created
> whenever a new stream gets created. This will avoid having to
> trigger primary surface creation from Qemu. And, this change
> also fixes the following error seen with v2:
> ../server/display-channel.cpp:2074:display_channel_create_surface:
> condition `!display->priv->surfaces[surface_id]' failed
> Rebase all patches on master
> 
> v2:
> 
> Update all patches to address review comments from Frediano
> Tested this series with msdkh264enc/dec plugins that will be added
> in another patch series
> 
> Michael Scherle (3):
>   Update spice-common submodule
>   gstreamer-encoder: Include dmabuf encoding conditionally for Linux
>   dcc-send: Fix freeze when video stream is stopped during ongoing draw
> 
> Vivek Kasireddy (7):
>   gstreamer-encoder: Use a h/w based encoder with Intel GPUs if possible
>     (v5)
>   dcc: Check to see if the client supports multiple codecs (v2)
>   dcc: Create a stream associated with gl_draw for non-gl clients (v6)
>   dcc-send: Encode and send gl_draw stream data to the remote client
>     (v4)
>   gstreamer-encoder: Add an encoder function that takes dmabuf fd as
>     input (v3)
>   video-stream: Don't stop a stream associated with gl_draw (v2)
>   gstreamer-encoder: Map the drm format to appropriate Gstreamer format
> 
>  configure.ac                     |   2 +-
>  meson.build                      |  12 +-
>  server/dcc-send.cpp              | 163 ++++++++++---
>  server/dcc.cpp                   |  31 ++-
>  server/dcc.h                     |   6 +
>  server/display-channel-private.h |   1 +
>  server/display-channel.cpp       |   1 +
>  server/gstreamer-encoder.c       | 389 ++++++++++++++++++++++++++++---
>  server/video-encoder.h           |  14 ++
>  server/video-stream.cpp          | 201 +++++++++++++---
>  server/video-stream.h            |   4 +
>  subprojects/spice-common         |   2 +-
>  12 files changed, 717 insertions(+), 109 deletions(-)
>