[Spice-devel] Stuttering video playback on LAN

[Victor Toso]

Hi,

On Mon, Jul 27, 2020 at 09:40:27AM +0200, Felix Leimbach wrote:
> Hi Uri
> 
> I've succeeded in using h264 with spice-streaming-agent, but
> only with a linux client. The windows client does not seem to
> support anything besides mjpeg, even after installing and
> tinkering with gstreamer.  I've opened a bugreport:
> 
> https://gitlab.com/virt-viewer/virt-viewer/-/issues/5

For windows, we use gstreamer for audio playback/record. I'd be
surprised if the spice-gtk code base used for linux would work
out-of-the box for windows on video decoding. So, afaics:

1) Remote-viewer will need to ship some extra plugins to handle
   video decoding (even mjpeg if you want to use gstreamer for
   that, as by default there is a mjepg-decoder in spice-gtk)

2) Some tweaking in spice-gtk to handle video with gstreamer plus
   tweaks to handle windows pipelines

If you want to work with h264, I think the problem could be
around license & patents.


> Another drawback: spice-streaming-agent only works with the
> x264enc encoder, which does not support VAAPI based hardware
> acceleration with the Intel GPU I passed through with GVT-g.
> 
> The vaapih264enc codec claimed that it cannot produce a x-h264
> stream:
> 
> # ./spice-streaming-agent -d -c gst.h264=vaapih264enc
> ...
> spice-streaming-agent[266317]: Gstreamer plugin: Specified encoder named 'vaapih264enc' cannot produce 'video/x-h264, stream-format=(string)byte-stream, framerate=(fraction)25/1' streams. Make sure that gst.CODEC=ENCODER is correctly specified and that the encoder is available.
> spice-streaming-agent[266317]: Gstreamer plugin: 'x264enc' encoder plugin is used
> 
> The avenc_h264 codec loaded but failed:
> 
> # ./spice-streaming-agent -c gst.h264=avenc_h264_omx:bitrate=100000
> spice-streaming-agent[269477]: Gstreamer plugin: Looking for encoder plugins which can produce a 'video/x-h264, stream-format=(string)byte-stream, framerate=(fraction)25/1' stream
> spice-streaming-agent[269477]: Gstreamer plugin: 'vaapih264enc' plugin is available
> spice-streaming-agent[269477]: Gstreamer plugin: 'x264enc' plugin is available
> spice-streaming-agent[269477]: Gstreamer plugin: 'avenc_h264_omx' plugin is available
> spice-streaming-agent[269477]: Gstreamer plugin: 'avenc_h264_omx' encoder plugin is used
> spice-streaming-agent[269477]: Gstreamer plugin: Trying to set encoder property: 'bitrate = 100000'
> ** (spice-streaming-agent:269477): CRITICAL **: 22:47:26.612: gst_vaapi_display_lock: assertion 'display != NULL' failed
> ** (spice-streaming-agent:269477): CRITICAL **: 22:47:26.612: gst_vaapi_display_unlock: assertion 'display != NULL' failed
> ** (spice-streaming-agent:269477): CRITICAL **: 22:47:26.617: gst_vaapi_display_lock: assertion 'display != NULL' failed
> ** (spice-streaming-agent:269477): CRITICAL **: 22:47:26.617: gst_vaapi_display_unlock: assertion 'display != NULL' failed
> spice-streaming-agent[269477]: No sample- EOS or state change
> 
> So I reverted back to using regular spice in qemu without
> spice-streaming-agent.
> I think it would be a huge improvement if the spice component
> in the qemu host process could leverage GPU based encoding with
> h264.

Yes,

	https://gitlab.freedesktop.org/spice/spice/-/blob/master/server/gstreamer-encoder.c#L899

> We wouldn't need a guest agent, we wouldn't require GPU
> passthrough and have great performance for multimedia
> use-cases. Not sure were I would open a feature request for
> that, though.

There are always people interested on spice server doing
streaming but it is complicated to accommodate in regards to
resources (it does not scale well in a host with multi VM).

This is/was primarily used in Xspice and x11spice for accessing
bare-metal OS.

Patches to improve this would be welcomed indeed.

Cheers,
Victor
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/spice-devel/attachments/20200731/03571eb3/attachment.sig>