[PATCH v8 0/6] dcc: Create a stream for non-gl/remote clients that want to use dmabuf (v8)

Tue Jun 18 07:09:49 UTC 2024

Hi Frediano,

> 
> Hi,
>    I spent quite some time this weekend trying to use this new series
> but I'm getting all blank screen for some reasons. Last time I got
> something so the encoder was working on the system.
Which encoder did you try with? x264enc or msdkh264enc?
If it is the latter, I am assuming you already have the following patch right:
https://gitlab.freedesktop.org/Vivek/spice/-/commit/34e342a2f7c2501d65d8cc9dd9def00f22fbd54a

Also, are you seeing any errors?

> I also had to do some changes for compatibility, most of the CI is not
> even compiling at the moment. Which distro are you using? I'll try to
> set up one similar.
Are you referring to the changes needed to use the udev changes from
spice-common?

As for my setup, I am using Fedora 39 on both Host and Guest but pretty
much all components associated with the Graphics stack and also the Host
and Guest kernels are compiled are pulled (and compiled) from their
respective repos (master branch). 

> I saw the changes in Qemu. As I said Qemu side cannot detect correctly
> if you are using a remote client or not, these patches are good for a
> time being but that's a wrong solution. Not that I have something
> better concrete at the moment, as I said I'm still trying this series.
I am using this Qemu branch to run the tests with this series:
https://gitlab.freedesktop.org/Vivek/qemu/-/commits/spice_gl_on_v4

And, here are the scenarios/tests I tried with this series, which all work as expected:
1) blob=true and virgl=false
./qemu-system-x86_64 -m 4096m -enable-kvm -cpu host -smp .....
-device virtio-vga,max_outputs=1,xres=1920,yres=1080,blob=true -spice port=3001,gl=on,disable-ticketing=on,preferred-codec=gstreamer:h264
-object memory-backend-memfd,id=mem1,size=4096M -machine memory-backend=mem1 -usb -device usb-tablet -serial stdio

In the above case, for blob=true to work, you need to make sure the Host kernel
includes support for udmabuf driver.

2) blob=false and virgl=false
./qemu-system-x86_64 -m 4096m -enable-kvm -cpu host -smp .....
-device virtio-vga,max_outputs=1,xres=1920,yres=1080 -spice port=3001,gl=on,disable-ticketing=on,preferred-codec=gstreamer:h264
-object memory-backend-memfd,id=mem1,size=4096M -machine memory-backend=mem1 -usb -device usb-tablet -serial stdio

In the above two cases, since virgl=false and there is no other GPU device available,
Gnome (in Guest) would automatically use the llvmpipe DRI driver (CPU based/software)
for rendering the desktop.

3) blob=false and virgl=true
 ./qemu-system-x86_64 -m 4096m -enable-kvm -cpu host -smp ......
-device virtio-vga-gl,max_outputs=1,xres=1920,yres=1080 -spice port=3001,gl=on,disable-ticketing=on,preferred-codec=gstreamer:h264
-object memory-backend-memfd,id=mem1,size=4096M -machine memory-backend=mem1 -usb -device usb-tablet -serial stdio

In all three cases above, I tested by booting to the Guest cmd line and launching Gnome + Xorg
by running the cmd startx. For testing Gnome + Wayland, I added WaylandEnable=True in
/etc/gdm/custom.conf and launched it by running systemctl start gdm.

Thanks,
Vivek

> 
> Regards,
>   Frediano
> 
> Il giorno lun 10 giu 2024 alle ore 20:03 Vivek Kasireddy
> <vivek.kasireddy at intel.com> ha scritto:
> >
> > For clients that cannot accept a dmabuf fd directly (such as those
> > running on a remote system), this patch series provides a way for
> > the Spice server to stream the gl/dmabuf data/buffer instead. This
> > is mostly done by enabling the creation of Gst memory using a dmabuf
> > fd as the source. This ability is useful given that dmabuf is the
> > standard mechanism for sharing buffers between various drivers and
> > userspace in many Graphics and Media usecases. Currently, this is
> > tested with Qemu and remote-viewer using the x264enc/avdec_h264
> > and msdkh264enc/dec plugins to stream the Guest/VM desktop but it
> > can be easily extended to other plugins and applications.
> >
> > Here is roughly how things work:
> > - The application (e.g, Qemu) chooses its preferred codec (a Gstreamer
> >   one) and calls gl_scanout (to update the fd) followed by gl_draw.
> > - In response, the Spice server checks to see if the client is capable
> >   of accepting a dmabuf fd directly or not. If yes, the fd is forwarded
> >   directly to the client; otherwise, a new stream is created.
> > - The Spice server then sends the dmabuf fd to the Gstreamer encoder
> >   which uses it as an input for creating an encoded buffer which is then
> >   sent to the client.
> > - Once the encoding process is done, an async completion cookie is sent
> >   to the application.
> >
> > Here is a link to the previous version that used a drawable to share
> > the dmabuf fd with the Gstreamer encoder:
> > https://lists.freedesktop.org/archives/spice-devel/2023-
> January/052948.html
> >
> > This version is tested together with following (required) patches in qemu:
> > https://gitlab.freedesktop.org/Vivek/qemu/-/commits/spice_gl_on_v4
> >
> > Changelog:
> >
> > v8:
> > - Added a new gstreamer-encoder patch to this series to convert drm
> format
> >   shared by the VMM to the appropriate Gstreamer format.
> >
> > v7:
> > - Revert back to the previous design where we do not share fd with the
> stream
> >   and scanout is the sole owner of the fd. This is because sharing fd
> ownership
> >   opens up a lot of corner cases.
> >
> > v6: (Frediano)
> > - Properly share ownership of the dmabuf fd between stream and scanout
> > - Ensure that a newly created stream is associated with all connected clients
> >
> > v5:
> > - Addressed review comments from Frediano mainly regarding adding
> autoconf
> >   support for gstreamer-allocators dependency and not needing to access
> >   scanout as part of gl draw operation
> >
> > v4:
> > - Test with Virgl enabled
> > - Associate dmabuf's y0_top with stream's top_down variable
> >
> > v3:
> > - Updated the second patch to have a new primary surface created
> >   whenever a new stream gets created. This will avoid having to
> >   trigger primary surface creation from Qemu. And, this change
> >   also fixes the following error seen with v2:
> >   ../server/display-channel.cpp:2074:display_channel_create_surface:
> >   condition `!display->priv->surfaces[surface_id]' failed
> > - Rebase all patches on master
> >
> > v2:
> > - Update all patches to address review comments from Frediano
> > - Tested this series with msdkh264enc/dec plugins that will be added
> >   in another patch series
> >
> > Cc: Frediano Ziglio <freddy77 at gmail.com>
> > Cc: Gerd Hoffmann <kraxel at redhat.com>
> > Cc: Marc-André Lureau <marcandre.lureau at redhat.com>
> > Cc: Dongwon Kim <dongwon.kim at intel.com>
> >
> > Vivek Kasireddy (6):
> >   dcc: Check to see if the client supports multiple codecs (v2)
> >   dcc: Create a stream associated with gl_draw for non-gl clients (v6)
> >   dcc-send: Encode and send gl_draw stream data to the remote client
> >     (v4)
> >   gstreamer-encoder: Add an encoder function that takes dmabuf fd as
> >     input (v3)
> >   gstreamer-encoder: Map the drm format to appropriate Gstreamer
> format
> >   video-stream: Don't stop a stream associated with gl_draw (v2)
> >
> >  configure.ac                     |   2 +-
> >  meson.build                      |   2 +-
> >  server/dcc-send.cpp              | 159 ++++++++++++++++----
> >  server/dcc.cpp                   |  31 ++--
> >  server/dcc.h                     |   6 +
> >  server/display-channel-private.h |   1 +
> >  server/display-channel.cpp       |   1 +
> >  server/gstreamer-encoder.c       | 246 ++++++++++++++++++++++++++-----
> >  server/video-encoder.h           |  14 ++
> >  server/video-stream.cpp          | 205 ++++++++++++++++++++++----
> >  server/video-stream.h            |   4 +
> >  11 files changed, 563 insertions(+), 108 deletions(-)
> >
> > --
> > 2.45.1
> >