Il giorno sab 15 apr 2023 alle ore 02:32 Kasireddy, Vivek
<vivek.kasireddy at intel.com> ha scritto:
>
> Hi Frediano,
>
> > Il giorno ven 31 mar 2023 alle ore 08:52 Kasireddy, Vivek
> > <vivek.kasireddy at intel.com> ha scritto:
> > >
> > > Hi Frediano,
> > >
> > > >
> > > > Il giorno lun 27 mar 2023 alle ore 07:53 Kasireddy, Vivek
> > > > <vivek.kasireddy at intel.com> ha scritto:
> > > > >
> > > > > Hi Frediano,
> > > > >
> > > > > > I was trying the patch series but the client keeps crashing. I
> > > > > > tried different versions of remote-viewer (from Fedora and from
> > > > > > Ubuntu) and they both crashed.
> > > > > > On host I installed Intel VA drivers , Gstreamer VAAPIs and forced
> > > > > > gstreamer:h264 as encoder.
> > > > > [Kasireddy, Vivek] I don't think any of these packages are needed if we
> > are
> > > > testing
> > > > > with a software (i.e, CPU-based) encoder/decoder (i.e, using
> > x264enc/x264dec
> > > > plugins).
> > > >
> > > > They should allow hardware encoding.
> > > [Kasireddy, Vivek] IIUC, currently, Spice only uses x264enc/x264dec plugins
> > and
> > > with these only software encoding is possible. We need to use different
> > plugins to
> > > enable hardware encoding. For instance we need to use either
> > vaapih264enc or
> > > msdkh264enc on Intel hardware. As I mentioned earlier, we are planning to
> > > add this support.
> > >
> > > >
> > > > > On my Fedora 37, these relevant plugins are provided by:
> > > > > # rpm -q --whatprovides /usr/lib64/gstreamer-1.0/libgstx264.so
> > > > > gstreamer1-plugins-ugly-1.20.5-1.fc37.x86_64
> > > > > # rpm -q --whatprovides /usr/lib64/gstreamer-1.0/libgstlibav.so
> > > > > gstreamer1-plugin-libav-1.20.5-1.fc37.x86_64
> > > > >
> > > > > The former provides x264enc and the latter provides x264dec. You can
> > either
> > > > choose
> > > > > the ones provided by your distro or build Gstreamer with -Dgst-plugins-
> > > > ugly:x264=enabled
> > > > > and -Dlibav=enabled. However, note that one of our eventual goals is to
> > cleanly
> > > > add a
> > > > > new Gstreamer pipeline to Spice to provide users an option for
> > hardware-based
> > > > > (i.e, GPU assisted) H.264 encoding/decoding using
> > msdkh264enc/msdkh264dec
> > > > plugins:
> > > > >
> > https://gstreamer.freedesktop.org/documentation/msdk/msdkh264enc.htm
> > l?gi-
> > > > language=c
> > > > >
> > > > > > Anything else you need to do?
> > > > > [Kasireddy, Vivek] Sorry, I was not expecting anyone would test the
> > Spice server
> > > > > patches and therefore did not provide all the required information to
> > test them.
> > > > > Anyway, I suspect the reason for your remote-viewer crash is because
> > there is no
> > > > > (primary) surface created. Here is the tentative Qemu patch that does
> > just that:
> > > > >
> > > >
> > > > Do you have a branch for Qemu somewhere?
> > > [Kasireddy, Vivek] Yes, I created one here:
> > > https://gitlab.freedesktop.org/Vivek/qemu/-/commits/pref_codec1
> > > https://gitlab.freedesktop.org/Vivek/spice/-/commits/encode_dmabuf1
> > >
> > > And, FYI, here is my exact qemu launch cmd (on Fedora 37):
> > > ./qemu-system-x86_64 -m 4096m -enable-kvm -cpu host -smp cores=8 -
> > drive file=./fed30kvm.img,format=qcow2,cache=none -vga none -device
> > e1000,netdev=net0,mac=DE:AD:BE:EF:BF:FA -netdev
> > tap,id=net0,ifname=tap0,script=no,downscript=no -device virtio-gpu-
> > pci,max_outputs=1,blob=true,xres=1920,yres=1080 -spice
> > port=3001,gl=on,disable-ticketing=on,preferred-codec=gstreamer:h264 -
> > object memory-backend-memfd,id=mem1,size=4096M -machine memory-
> > backend=mem1 -usb -device usb-tablet -serial stdio
> > >
> > > >
> > > > > fd = egl_get_fd_for_texture(ssd->ds->texture,
> > > > >
> > > > > I did not yet post this patch to the Qemu mailing list because I still need
> > to
> > > > implement
> > > > > feature negotiation between Qemu and Spice to auto-detect this
> > capability. And,
> > > > > in addition to this patch, you also need the patch that adds the
> > "preferred-codec"
> > > > > option to Qemu, unless you are hardcoding it in Spice:
> > > > > https://lists.nongnu.org/archive/html/qemu-devel/2023-
> > 01/msg04999.html
> > > > >
> > > >
> > > > Yes, I hardcoded that in SPICE. Note that I posted a comment in that
> > > > commit, I think it's valid as it is.
> > > > Also I think it should not change the SPICE default.
> > > [Kasireddy, Vivek] Yes, I noted your comment; I'll make the change in the
> > > next version.
> > >
> > > Thanks,
> > > Vivek
> > >
> > > >
> > > > > Lastly, not sure if it makes a difference, but here are the relevant
> > options I am
> > > > > using to launch Qemu:
> > > > > -device virtio-gpu-pci,max_outputs=1,blob=true,xres=1920,yres=1080
> > > > > -spice port=3001,gl=on,disable-ticketing=on,preferred-
> > codec=gstreamer:h264
> > > > > -object memory-backend-memfd,id=mem1,size=4096M
> > > > > -machine memory-backend=mem1 -usb -device usb-tablet -serial stdio
> > > > >
> > > > > Thanks,
> > > > > Vivek
> > > > >
> > > >
> > > > I will try again.
> > > >
> > > > Thanks,
> > > > Frediano
> > > >
> > > > > >
> > > > > > On the logs I find:
> > > > > >
> > > > > > 2023-03-25T19:31:36.034007Z qemu-system-x86_64: warning: Spice:
> > > > > > ../server/dcc-send.cpp:1786:red_marshall_gl_draw_stream: bad
> > return
> > > > > > value (0) from VideoEncoder::encode_dmabuf
> > > > > > 2023-03-25T19:31:37.064219Z qemu-system-x86_64: warning: spice:
> > no
> > > > > > gl-draw-done within one second
> > > > > > 2023-03-25T19:31:58.214387Z qemu-system-x86_64: warning: Spice:
> > > > > > Connection reset by peer
> > > > > > 2023-03-25T19:31:58.214482Z qemu-system-x86_64: warning: Spice:
> > > > > > Connection reset by peer
> > > > > > 2023-03-25T19:31:58.214580Z qemu-system-x86_64: warning: Spice:
> > > > > > Connection reset by peer
> > > > > > 2023-03-25T19:31:58.214642Z 2023-03-25T19:31:58.214636Z
> > > > > > qemu-system-x86_64:qemu-system-x86_64: warning: Spice:
> > Connection
> > > > > > reset by peer
> > > > > > warning: Spice: display:0 (0x55947e76fd10): Connection reset by peer
> > > > > > 2023-03-25T19:31:58.214721Z qemu-system-x86_64: warning: Spice:
> > > > > > Connection reset by peer
> > > > > > 2023-03-25T19:31:58.214841Z qemu-system-x86_64: warning: Spice:
> > > > > > Connection reset by peer
> > > > > > 2023-03-25T19:31:58.215057Z qemu-system-x86_64: warning: Spice:
> > > > > > Connection reset by peer
> > > > > >
> > > > > > 0 value from encode_dmabuf should mean
> > > > VIDEO_ENCODER_FRAME_DROP.
> > > > > >
> > > > > > Regards,
> > > > > > Frediano
> > > > > >
> >
> > Hi,
> > some updates. I grab your exact versions and compiled them again.
> > At the beginning I still had some problems with codecs and security
> > settings (seccomp was enabled causing Qemu to crash on gstreamer code
> > ,turned off).
> > However even after solving them the result is not that great. The
> > video is not fluid and when there are some updates it is almost
> > "garbage". Which type of GPU are you using? I'm using integrated Intel
> > and I remember they have a not-linear memory addressing for textures
> > which lead to these garbaged style images due to trying to interpret
> > memory as linear.
> [Vivek] I am using an integrated Intel GPU (Gen 12, codenamed TigerLake/AlderLake)
> as well. Regardless, IIUC, the memory backing the dmabuf fd is linear as Virtio-gpu
> does not allow/expose modifier support -- which is needed to allocate
> non-linear/tiled memory. The video not being fluid issue is probably a symptom
> of the time it (software encoder/x264enc) takes to encode a frame. In my
> setup, it takes about ~10-12 ms to encode a 1920x1080 BGRX frame -- which
> obviously drops the FPS to ~25-30. And, the "garbage" behavior could most
> likely be environmental. In other words, in this use-case, we are heavily
> relying on the complex Gstreamer stack which includes 100s of libraries and
> plugins; so, any of these could be playing a role. Can you please try building
> the Gstreamer stack yourself and test to see if it helps? Also, can you forcefully
> use only the avdec_h264 decoder (by possibly hiding/removing libgstvaapi.so,
> libgstva.so, libgstopenh264.so and other h264 decoders) to see if the garbage
> goes away?
>
It was more that I see 3/4 changes for half a second then 4/5 seconds
no changes than not much fluid.
About not linear I remember that trying to mmap the fd in the past I
got no-linear image and what I'm getting is pretty similar.
Can you dump the 2 GStreamer pipelines (Qemu and client side)
https://gstreamer.freedesktop.org/documentation/tutorials/basic/debugging-tools.html?gi-language=c#getting-pipeline-graphs
(probably you know but I never remember the environment).
> Lastly, could you please share details about your setup such as OS version,
> which libraries you have built (decoders?) and installed, Qemu/Spice build
> options, runtime options, etc., so that I can try reproducing this issue in
> my setup?
>
I tried to not change much the system, it's an Ubuntu 22.04.2 LTS,
card (integrated)
00:02.0 VGA compatible controller: Intel Corporation CometLake-U GT2
[UHD Graphics] (rev 02) (prog-if 00 [VGA controller])
Subsystem: Lenovo CometLake-U GT2 [UHD Graphics]
Flags: bus master, fast devsel, latency 0, IRQ 158, IOMMU group 1
Memory at e9000000 (64-bit, non-prefetchable) [size=16M]
Memory at a0000000 (64-bit, prefetchable) [size=512M]
I/O ports at 4000 [size=64]
Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
Capabilities: <access denied>
Kernel driver in use: i915
Kernel modules: i915
gstreamer came from packages, Qemu and SPICE are from your branches.
I tried different remote-viewer, eventually I found an issue with
initialization so I had to change a line of code in spice-gtk.
> > I also had to change spice-gtk due to a missing specification in the
> > encoder. Which client did you use? Which version? Maybe the bug was
> > introduced later.
> [Vivek] I am currently using remote-viewer version 11.0-5.fc37 as the
> client but I have also successfully tested with the remote-viewer version
> from Fedora 36. In addition, I have also tested with spice-gtk master as well.
>
Maybe the plugin you are using is not having that issue, the graph will help.
> > Also I have some warnings in Qemu logs due to attempts to create
> > primary surfaces multiple times.
> [Vivek] Yeah, I see them as well. I'll eventually fix them as they seem
> harmless so far.
>
At least it should cause a memory leak. But it is not good.
> >
> > By the way, the code looks better than the previous version but as
> > commented the running behaviour is not that great.
> [Vivek] As I mentioned above, I think there is not much we can do in Spice
> to fix the behavior you are seeing given the heavy reliance on Gstreamer
> in this use-case. However, we are also actively working on cleanly adding
> Spice options to enable hardware encoders/decoders for Intel GPUs to
> reduce the encode time to < 5 ms. In addition, we are also trying to
> identify and fix all the other performance bottlenecks. We'll post patches
> as soon as we clean everything up and fix all the issues we are seeing.
>
> > The style (well, also the style of spice code I have to admit) is a
> > bit "messy" but better to fix the big problems first than try to solve
> [Vivek] Ok, I guess we can fix it incrementally if you can first point out
> the worst style violoations.
>
I'll try to write some fixup commits and post some comments to some
"code smell".
> Thanks,
> Vivek
>
> > everything at once.
> >
> > Regards,
> > Frediano
> >
> > > > > > Il giorno gio 16 mar 2023 alle ore 06:05 Vivek Kasireddy
> > > > > > <vivek.kasireddy at intel.com> ha scritto:
> > > > > > >
> > > > > > > For clients that cannot accept a dmabuf fd directly (such as those
> > > > > > > running on a remote system), this patch series provides a way for
> > > > > > > the Spice server to stream the gl/dmabuf data/buffer instead. This
> > > > > > > is mostly done by enabling the creation of Gst memory using a
> > dmabuf
> > > > > > > fd as the source. This ability is useful given that dmabuf is the
> > > > > > > standard mechanism for sharing buffers between various drivers
> > and
> > > > > > > userspace in many Graphics and Media usecases. Currently, this is
> > > > > > > only used/tested with Qemu and remote-viewer using the
> > x264enc/dec
> > > > > > > codec to stream the Guest/VM desktop but it can be easily extended
> > > > > > > to other plugins and applications.
> > > > > > >
> > > > > > > Here is roughly how things work:
> > > > > > > - The application (e.g, Qemu) chooses its preferred codec (a
> > Gstreamer
> > > > > > > one) and calls gl_scanout (to update the fd) followed by gl_draw.
> > > > > > > - In response, the Spice server checks to see if the client is capable
> > > > > > > of accepting a dmabuf fd directly or not. If yes, the fd is forwarded
> > > > > > > directly to the client; otherwise, a new stream is created.
> > > > > > > - The Spice server then sends the dmabuf fd to the Gstreamer
> > encoder
> > > > > > > which uses it as an input for creating an encoded buffer which is
> > then
> > > > > > > sent to the client.
> > > > > > > - Once the encoding process is done, an async completion cookie is
> > sent
> > > > > > > to the application.
> > > > > > >
> > > > > > > Here is a link to the previous version that used a drawable to share
> > > > > > > the dmabuf fd with the Gstreamer encoder:
> > > > > > > https://lists.freedesktop.org/archives/spice-devel/2023-
> > January/052948.html
> > > > > > >
> > > > > > > Cc: Frediano Ziglio <freddy77 at gmail.com>
> > > > > > > Cc: Gerd Hoffmann <kraxel at redhat.com>
> > > > > > > Cc: Marc-André Lureau <marcandre.lureau at redhat.com>
> > > > > > > Cc: Dongwon Kim <dongwon.kim at intel.com>
> > > > > > >
> > > > > > > Vivek Kasireddy (5):
> > > > > > > dcc: Check to see if the client supports multiple codecs
> > > > > > > dcc: Create a stream associated with gl_draw for non-gl clients
> > > > > > > dcc-send: Encode and send gl_draw stream data to the remote
> > client
> > > > > > > gstreamer-encoder: Add an encoder function that takes dmabuf fd
> > as
> > > > > > > input
> > > > > > > video-stream: Don't stop a stream if a gl_draw operation is
> > pending
> > > > > > >
> > > > > > > meson.build | 2 +-
> > > > > > > server/dcc-private.h | 4 ++
> > > > > > > server/dcc-send.cpp | 89 ++++++++++++++++++++++-
> > > > > > > server/dcc.cpp | 36 +++++++---
> > > > > > > server/display-channel-private.h | 6 ++
> > > > > > > server/gstreamer-encoder.c | 119
> > > > ++++++++++++++++++++++++++++++-
> > > > > > > server/video-encoder.h | 13 ++++
> > > > > > > server/video-stream.cpp | 65 ++++++++++++++++-
> > > > > > > server/video-stream.h | 2 +
> > > > > > > 9 files changed, 319 insertions(+), 17 deletions(-)
> > > > > > >
> > > > > > > --
> > > > > > > 2.37.2
> > > > > > >