[Spice-devel] [PATCH 0/11] Add GStreamer and VP8 support

[Francois Gouget]

This is a followup on Jeremy White's concept patch to use GStreamer:
http://www.spinics.net/lists/spice-devel/msg10030.html

There is still a lot of work ahead but this patches series has all the 
infrastructure so we're sending it out to verify that it looks ok.

First, with this patch series:
 1. You can use the SpiceVideoCodecs option in the xorg.conf file (or
    $XSPICE_VIDEO_CODECS) to set your video encoder and codec 
    preferences. The default is still the builtin MJPEG encoder.

 2. You can do the same from QEmu by setting the video-codecs option.

 3. Clients can expose new display capabilities to advertise that 
    they support either or both of the MJPEG and VP8 codecs. The new 
    server will automatically pick a codec they support based on the 
    above preference list. Older clients that don't advertise these new 
    capabilities will automatically get an MJPEG stream. So old and new 
    clients and servers should all be compatible.

 4. The GTK and and HTML5 clients have been updated to support both 
    MJPEG and VP8.

 5. Using GStreamer as the video encoder should work well for either 
    codecs as long as the stream does not hit your network connection's 
    bandwidth limit. Essentially this means it should be ok on LANs and 
    fast WiFi networks. This should even work with multiple clients.


There's one known issue in the framework: if the Spice server and 
clients have no codec in common, red_display_create_stream() will set 
video_encoder to NULL and that's not handled well. Currently this should 
really not happen as all clients support MJPEG and we have the builtin 
MJPEG encoder as a fallback. The root of the issue is that the 
red_display_create_stream() callers assume it will never fail. But I 
don't know how to change that yet.


The GStreamer backend still has a number of limitations:
 1. It's still based on GStreamer 0.10. I don't think moving to 1.0 
    will be a problem, if anything it should help (famous last words).
    For instance support for dynamically changing the bitrate appears to 
    be better in 1.0.

 2. It still does not have any rate control but should work fine as long 
    as the bandwidth is sufficient. The issue with rate control is that, 
    at least in 0.10, GStreamer encoders don't like bitrate changes, 
    need some time after each change to actually match the new bitrate, 
    still tend to exceed it from time to time, and sometimes just won't 
    get anywhere near it. Part of these issues are likely to be bugs in 
    the current implementation and it should be possible to work around 
    most others.


Feedback would be greatly appreciated.


Cheers,

-- 
Francois Gouget <fgouget at codeweavers.com>