[Spice-devel] [spice v12 00/26] Add GStreamer support for video streaming

[Francois Gouget]

This patch series adds support for using GStreamer to encode and 
decode the video streams, adding support for VP8 and h264 codecs.

As before the patches can also be grabbed from the gst branch of the
repositories below:

spice:          https://github.com/fgouget/spice
spice-gtk:      https://github.com/fgouget/spice-gtk
xf86-video-qxl: https://github.com/fgouget/xf86-video-qxl
spice-protocol: https://github.com/fgouget/spice-protocol

See also gst-sync for the old spice-gtk code. (there's also 'extras'
branches with more experimental/future patches for the curious)


Besides rebases, the changes from v11 are limited to spice-gtk. 
Specifically they fix two compatibility issues:
 * V11 used a GAsyncQueue and relied on g_async_queue_push_front() which 
   is only available in GLib >= 2.46 which is unfortunately not 
   available in Debian 8 and presumably on many other Linux 
   distributions.
 * It also associated the original network message with the 
   corresponding GStreamer buffer using the GstMeta API. This is 
   necessary in order to know where to display the decoded frame as this 
   could change from one frame to the next. However some GStreamer 
   elements have (refcounting) bugs that caused the metadata to be lost 
   in the pipeline. I had a workaround in place that worked on Debian 
   Testing but did not on Debian 8.
   https://bugzilla.gnome.org/show_bug.cgi?id=757254

So in the end I reworked both by putting the metadata in a regular 
GQueue and using the PTS timestamp to match elements in that queue to 
GStreamer's buffers.


Patches and changes from v10:
  server: Check the client video codec capabilities
      Fixed the case where the server and client cannot agree on the 
      video codec to use.
  server: Let the administrator pick the video encoder and codec
      The video_codecs GArray is now considered immutable which avoids 
      copying around and avoids having to make the
      RED_WORKER_MESSAGE_SET_VIDEO_CODECS RPC synchronous.
  server: Use the optimal number of threads for VP8 encoding
      Using the optimal number of cores for the VP8 encoder is now in a 
      separate patch (though skipping it will cause conflicts with 
      patches 14 and 16).
  server: Avoid copying the input frame in the GStreamer encoder
      Changed the zero-copy approach to not require ref/unref() of 
      RedDrawable objects to be thread-safe. Thread-safety aspects are 
      instead handled in gstreamer-encoder.c now.
  server: Adjust the frame rate based on the GStreamer encoding time
      This patch limits the frame rate (i.e. drops frames) when the 
      server has trouble keeping up with the encoding. This code seems 
      to only be needed in SpiceDeferredFPS mode (so maybe it's in the
      SpiceDeferredFPS code that something should be modified). In 
      any case this patch can be skipped without any impact on the rest 
      of the series.
  server: Respect the GStreamer encoder's valid bit rate range
      set_gstenc_bitrate() should now pass the right integer type to 
      g_object_set().
  server: Add support for GStreamer 0.10
      Reduced the number of #ifdefs for GStreamer 0.10 using 
      Christophe's suggestions.
  spice-gtk: Add a GStreamer video decoder for MJPEG, VP8 and h264
    - Tweaked some function names in the client's GStreamer decoder to 
      avoid potential name conflicts.
    - Fixed a race condition in the client which would cause it to 
      freeze when hovering madly over the seek barin YouTube videos.
    - As a side effect the new code now schedules the frames rending 
      itself so it can adapt when the mm-time get yanked around.
    - It also does not queue the frames in the pipeline and thus is less 
      likely to lose them when size changes force us to rebuild it.
  spice-gtk: Avoid GAsyncQueue for compatibility with GLib < 2.46
      This patch is optional and avoids the dependency on 
      g_async_queue_push_front() which is missing in GLib < 2.46 
      (e.g. for Debian 8).

-- 
2.8.0.rc3