[Spice-devel] Stuttering video playback on LAN

Tue Jul 28 11:40:27 UTC 2020

Hi Frediano,

yes, libva-intel-driver is installed and I tested vaapi successfully playing mp4 files with mpv and vlc.
The vaainfo command also showed that vaapi is working.

Probably, as Kevin pointed out, there's some other issue with my gestreamer.
I'm not investigating this though, since the windows virt-viewer build does not support h264 at all. If that gets fixed I'll give it another try.

Appreciate your input though, thanks!

Felix

On 28.07.20 13:17, Frediano Ziglio wrote:
> Hi,
>    did you install libva-intel-driver package?
> Without it vaapi won't work.
>
> Frediano
>
> > 
> > Hi Uri
> > 
> > I've succeeded in using h264 with spice-streaming-agent, but only with a
> > linux client. The windows client does not seem to support anything besides
> > mjpeg, even after installing and tinkering with gstreamer.
> > I've opened a bugreport:
> > https://gitlab.com/virt-viewer/virt-viewer/-/issues/5
> > 
> > Another drawback: spice-streaming-agent only works with the x264enc encoder,
> > which does not support VAAPI based hardware acceleration with the Intel GPU
> > I passed through with GVT-g.
> > 
> > The vaapih264enc codec claimed that it cannot produce a x-h264 stream:
> > 
> > # ./spice-streaming-agent -d -c gst.h264=vaapih264enc
> > ...
> > spice-streaming-agent[266317]: Gstreamer plugin: Specified encoder named
> > 'vaapih264enc' cannot produce 'video/x-h264,
> > stream-format=(string)byte-stream, framerate=(fraction)25/1' streams. Make
> > sure that gst.CODEC=ENCODER is correctly specified and that the encoder is
> > available.
> > spice-streaming-agent[266317]: Gstreamer plugin: 'x264enc' encoder plugin is
> > used
> > 
> > The avenc_h264 codec loaded but failed:
> > 
> > # ./spice-streaming-agent -c gst.h264=avenc_h264_omx:bitrate=100000
> > spice-streaming-agent[269477]: Gstreamer plugin: Looking for encoder plugins
> > which can produce a 'video/x-h264, stream-format=(string)byte-stream,
> > framerate=(fraction)25/1' stream
> > spice-streaming-agent[269477]: Gstreamer plugin: 'vaapih264enc' plugin is
> > available
> > spice-streaming-agent[269477]: Gstreamer plugin: 'x264enc' plugin is
> > available
> > spice-streaming-agent[269477]: Gstreamer plugin: 'avenc_h264_omx' plugin is
> > available
> > spice-streaming-agent[269477]: Gstreamer plugin: 'avenc_h264_omx' encoder
> > plugin is used
> > spice-streaming-agent[269477]: Gstreamer plugin: Trying to set encoder
> > property: 'bitrate = 100000'
> > ** (spice-streaming-agent:269477): CRITICAL **: 22:47:26.612:
> > gst_vaapi_display_lock: assertion 'display != NULL' failed
> > ** (spice-streaming-agent:269477): CRITICAL **: 22:47:26.612:
> > gst_vaapi_display_unlock: assertion 'display != NULL' failed
> > ** (spice-streaming-agent:269477): CRITICAL **: 22:47:26.617:
> > gst_vaapi_display_lock: assertion 'display != NULL' failed
> > ** (spice-streaming-agent:269477): CRITICAL **: 22:47:26.617:
> > gst_vaapi_display_unlock: assertion 'display != NULL' failed
> > spice-streaming-agent[269477]: No sample- EOS or state change
> > 
> > So I reverted back to using regular spice in qemu without
> > spice-streaming-agent.
> > I think it would be a huge improvement if the spice component in the qemu
> > host process could leverage GPU based encoding with h264. We wouldn't need a
> > guest agent, we wouldn't require GPU passthrough and have great performance
> > for multimedia use-cases. Not sure were I would open a feature request for
> > that, though.
> > 
> > Best,
> > Felix
> > 
> > 
> > On 18.05.20 12:21, Uri Lublin wrote:
> > > On 5/17/20 6:35 PM, Felix Leimbach wrote:
> > >> Hi Uri,
> > >>
> > >> On 17.05.20 16:08, Uri Lublin wrote:
> > >>> On 5/16/20 7:07 PM, Felix Leimbach wrote:
> > >>>>
> > >>>>>>
> > >>>>>> I experience stuttering video playback in remote-viewer despite
> > >>>>>> connecting
> > >>>>>> via GBit/s LAN, using fast hardware and the QXL driver.
> > >>>>>> Up until a video size of roughly 800x600 the playback is smooth. But
> > >>>>>> on
> > >>>>>> anything bigger, like my native resolution of 2540x1440, video
> > >>>>>> playback is
> > >>>>>> stuttering annoyingly.
> > >>>>>> After lots of unsuccessful tinkering with spice parameters and qxl
> > >>>>>> parameters
> > >>>>>> I'm asking you guys for help.
> > >>>>>>
> > >>>>>> Client:
> > >>>>>> Windows 10 1909
> > >>>>>> Remote Viewer 8.0-256
> > >>>>>> Quadcore i7-7820HQ 2.9GHz
> > >>>>>> 16GB DDR4 RAM
> > >>>>>>
> > >>>>>> Host of the VM:
> > >>>>>> Gentoo Linux
> > >>>>>> Kernel 4.14.172
> > >>>>>> Qemu 4.2.0
> > >>>>
> > >>>> I've updated to newer versions in the meantime, but no noticeable
> > >>>> changes.
> > >>>>
> > >>>> Qemu 5.0.0
> > >>>> Host kernel 5.4.39
> > >>>> Remote Viewer 9.0-256 (x64) on the Windows 10 Client
> > >>>>
> > >>>>>> <snipped>
> > >>>>>>
> > >>>
> > >>> <snipped>
> > >>>
> > >>>> I noticed very high CPU usage in the guest during playback, because
> > >>>> chrome, vlc, mpv used software h264 decoding.
> > >>>> I fixed this by passing a virtualized instance of the hosts Intel GPU to
> > >>>> the guest via GVT-g.
> > >>>>
> > >>>> These are the qemu parameters I use for GVT-g:
> > >>>> -spice port=5906,addr=10.42.2.250,password=changed
> > >>>> -vga virtio
> > >>>> -display egl-headless,rendernode=/dev/dri/card0
> > >>>> -device
> > >>>> vfio-pci,sysfsdev=/sys/bus/mdev/devices/f14c80d5-9ade-4802-9509-1d877d32d159,display=on,ramfb=on,driver=vfio-pci-nohotplug
> > >>>
> > >>> Perhaps here it would help to set streaming-video=all -spice option.
> > >>
> > >> It doesn't really help. Still very stuttering playback, even with a
> > >> video-player window size of only 720x576.
> > >> I now believe this is caused by a CPU bottleneck on the CLIENT (not
> > >> guest), i.e. my Windows 10 machine.
> > >> virt-viewer.exe uses consistently 12% CPU, which is 100% of one core (Quad
> > >> Core with SMT => 8 logical cores).
> > >> This is crazy, since the CPU core is running a modern i7 at 3.6 GHz
> > >> (Turbo) with no other significant CPU users (system is idle).
> > >> Could this be a problem of remote-viewer.exe not using gstreamer codecs
> > >> (and hw accel) properly?
> > >> Got these spice-debug messages on the client: "no video decoders from
> > >> GSTreamer for {mjpeg,vp8,h264,vp9,h265} were found".
> > >
> > > Maybe the client is built without gstreamer.
> > 
> > 
> > 
> > 
> > >> Also I noticed that the qemu process on the host is using much CPU,
> > >> probably due to spice encoding the video frames.
> > >> The host shows 4 qemu-ystem-x86_64 threads using 93%, 34%, 24%, 16% CPU
> > >> when playing a 720x576 video in the guest.
> > >> However the guest shows only 14% and 7% use for its 2 vCPUs in htop
> > >> because it uses VA-API (GPT-g) in the video player (vlc, mpv).
> > >
> > > With egl-headless, spice-server encodes the whole screen, not 720x576
> > > video. Since the client
> > > supports almost no codec likely
> > > it's using mjpeg
> > >
> > >>
> > >> So the host qemu threads use MUCH more CPU than the guest. When I
> > >> disconnect spice (video still playing) the host qemu CPU usage drops to
> > >> 35%, 1%, 1%, 1%, as expected.
> > >> Conclusion: Spice encoding on the host is very CPU hungry.
> > >
> > > Your conclusion makes sense to me.
> > >
> > >>
> > >> Is it possible to GPU-accelerate the video encoding in the host qemu
> > >> process?
> > >
> > > I think it can not use anything other than mjpeg if that's the only codec
> > > the client supports.
> > >
> > >> I've built spice and qemu with gstreamer and drm support and VA-API is
> > >> working nicely.
> > >> Can spice use that? How?
> > >
> > > Possibly, for H264 you need to change the get_gst_codec_name (gstreamer
> > > pipe);
> > > For vp8 it should be done already.
> > >
> > >>
> > >>>>
> > >>>> Unfortunately video playback is still not smoother. In fact it is about
> > >>>> the same smoothness but new visual artefacts in the video make it
> > >>>> worse. I think this is due to egl-headless.
> > >>>> For testing/comparison I installed a Windows 10 guest with the same
> > >>>> GVT-g GPU and used RDP with h264 activated. Playback was much better
> > >>>> and used only about 120MBit/s.
> > >>>>
> > >>>> Next I tried using the spice-streaming-agent in the guest to send a h264
> > >>>> encoded picture via spice.
> > >>>> However, the windows build of remote-viewer doesn't seem to support
> > >>>> this. The new spice display is created and I see the mouse cursor in it
> > >>>> but no picture (just black).
> > >>>
> > >>> You may be the first to test spice-streaming-agent + windows client.
> > >>>
> > >>> I think there is some work to be done in the windows client to make
> > >>> it handle better streams from spice-streaming-agent
> > >>>
> > >>> Also there is some work to be done to enable spice-streaming-agent
> > >>> on windows guests.
> > >>>
> > >>>> Log from the guest:
> > >>>> felix at idefix:~$ ./spice-streaming-agent -d
> > >>>> spice-streaming-agent[2465]: GOT START_STOP message -- request to START
> > >>>> streaming
> > >>>> spice-streaming-agent[2465]: streaming starts now
> > >>>> spice-streaming-agent[2465]: Got device info of 1 devices from the
> > >>>> plugin
> > >>>> spice-streaming-agent[2465]:    stream id 0: device address:
> > >>>> pci/0000/06.0, device display id: 2
> > >>>> spice-streaming-agent[2465]: got a frame -- size is 321265 (26 ms)
> > >>>> (1589641660285 ms from last frame)(1589641660258136 us)
> > >>>> spice-streaming-agent[2465]: wXh 1920X1200  codec=1
> > >>>
> > >>> Note that it's not H264, but MJPEG (codec=1)
> > >>
> > >> You are right, so I debugged this:
> > >>
> > >> Launching spice-streaming-agent with GST_DEBUG=6 shows a bunch of h264
> > >> related messages which seem to indicate that the gstreamer codecs are
> > >> loaded (not sure):
> > >> gstregistry.c:461:gst_registry_add_plugin:<registry0> adding plugin
> > >> 0x55d279f8fe70 for filename
> > >> "/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstuvch264.so"
> > >> gstregistry.c:577:gst_registry_add_feature:<registry0> adding feature
> > >> 0x55d279fe1650 (video/x-h264)
> > >> gstregistrychunks.c:729:gst_registry_chunks_load_feature: Added feature
> > >> vp8enc, plugin 0x55d279ed1150 vpx
> > >>
> > >> However I didn't find an argument to force spice-streaming-agent to use a
> > >> specific codec. Reading the source code I found the option "-c
> > >> codec_name=h264", but it doesn't seem to have any effect.
> > >>
> > >> Maybe it fails auto-negotiation with the client which does not support it?
> > >
> > > Likely.
> > >
> > > Uri.
> > >
> > >> On the client with the "--spice-debug" option I found interesting
> > >> messages:
> > >>
> > >> (remote-viewer.exe:15956): Spice-DEBUG: 17:17:06.218:
> > >> ../src/channel-display-gst.c:792:gstvideo_has_codec: No video decoders
> > >> from GStreamer for mjpeg were found
> > >> (remote-viewer.exe:15956): GSpice-DEBUG: 17:17:06.218:
> > >> ../src/channel-display.c:894 GStreamer does not support the mjpeg codec
> > >> (remote-viewer.exe:15956): Spice-DEBUG: 17:17:06.219:
> > >> ../src/channel-display-gst.c:792:gstvideo_has_codec: No video decoders
> > >> from GStreamer for vp8 were found
> > >> (remote-viewer.exe:15956): GSpice-DEBUG: 17:17:06.220:
> > >> ../src/channel-display.c:894 GStreamer does not support the vp8 codec
> > >> (remote-viewer.exe:15956): Spice-DEBUG: 17:17:06.221:
> > >> ../src/channel-display-gst.c:792:gstvideo_has_codec: No video decoders
> > >> from GStreamer for h264 were found
> > >> (remote-viewer.exe:15956): GSpice-DEBUG: 17:17:06.222:
> > >> ../src/channel-display.c:894 GStreamer does not support the h264 codec
> > >> (remote-viewer.exe:15956): Spice-DEBUG: 17:17:06.223:
> > >> ../src/channel-display-gst.c:792:gstvideo_has_codec: No video decoders
> > >> from GStreamer for vp9 were found
> > >> (remote-viewer.exe:15956): GSpice-DEBUG: 17:17:06.224:
> > >> ../src/channel-display.c:894 GStreamer does not support the vp9 codec
> > >> (remote-viewer.exe:15956): Spice-DEBUG: 17:17:06.227:
> > >> ../src/channel-display-gst.c:792:gstvideo_has_codec: No video decoders
> > >> from GStreamer for h265 were found
> > >> (remote-viewer.exe:15956): GSpice-DEBUG: 17:17:06.230:
> > >> ../src/channel-display.c:894 GStreamer does not support the h265 codec
> > >>
> > >> Any ideas how to fix this? Should I open a bug report >
> > >>>> <snipped>
> > >>>>
> > >>>> If I use remote viewer from a linux client then it does indeed work!
> > >>>> Playback is nearly smooth, about the same as with RDP and h264!
> > >>>> So I guess it's a bug in the windows remote-viewer. Seems like it
> > >>>> doesn't have gstreamer support, so I'll open a bug report.
> > >>>>
> > >>>> Any other ideas what I can try to get good reasonable video playback
> > >>>> with good office-work performance?
> > >>>>
> > >>>> On a side note: Audio via spice isn't working. I hear a few strange
> > >>>> noises and then only silence. So I use pulseaudio transmitting the
> > >>>> sound to the client independent of spice.
> > >>
> > >> Found the reason: It's a regression in the windows version of
> > >> remote-viewer. Was caused by libgstdirectsound.dll being replaced with
> > >> libgstwasapi.dll and this changed the audio subsystem being used.
> > >> Details: https://gitlab.com/virt-viewer/virt-viewer/-/issues/2
> > >>
> > >> Cheers,
> > >> Felix
> > >>
> > 
> > _______________________________________________
> > Spice-devel mailing list
> > Spice-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/spice-devel
> > 
>
-- 
Beste Grüße
Felix Leimbach