[Spice-devel] [spice v11 00/27] Add GStreamer support for video streaming

[Francois Gouget]

This patch series adds support for using GStreamer to encode and decode 
the video streams, adding support for VP8 and h264 codecs.

As before the patches can also be grabbed from the gst branch of the
repositories below:

spice:          https://github.com/fgouget/spice
spice-gtk:      https://github.com/fgouget/spice-gtk
xf86-video-qxl: https://github.com/fgouget/xf86-video-qxl
spice-protocol: https://github.com/fgouget/spice-protocol

See also gst-sync for the old spice-gtk code. (there's also 'extras'
branches with more experimental/future patches for the curious)


Patches and changes from v10:

Francois Gouget (26):
  server: Enable adding alternative MJPEG video encoders
  server: Add a GStreamer 1.0 MJPEG video encoder and use it by default
  server: Check the client video codec capabilities
      Fixed the case where the server and client cannot agree on the 
      video codec to use.
  server: Let the administrator pick the video encoder and codec
      The video_codecs GArray is now considered immutable which avoids 
      copying around and avoids having to make the
      RED_WORKER_MESSAGE_SET_VIDEO_CODECS RPC synchronous.
  server: Add VP8 support to the GStreamer video encoder
  server: Use the optimal number of threads for VP8 encoding
      Using the optimal number of cores for the VP8 encoder is now in a 
      separate patch (though skipping it will cause conflicts with 
      patches 14 and 16).
  server: Let the video encoder manage the compressed buffer
  server: Avoid copying the input frame in the GStreamer encoder
      Changed the zero-copy approach to not require ref/unref() of 
      RedDrawable objects to be thread-safe. Thread-safety aspects are 
      instead handled in gstreamer-encoder.c now.
  server: Add h264 support to the GStreamer video encoder
  server: Shape the bit rate of the GStreamer codecs output
  server: Adjust the GStreamer encoder bit rate to the network
  server: Adjust the frame rate based on the GStreamer encoding time
      This patch limits the frame rate (i.e. drops frames) when the 
      server has trouble keeping up with the encoding. This code seems 
      to only be needed in SpiceDeferredFPS mode (so maybe it's in the
      SpiceDeferredFPS code that something should be modified). In 
      any case this patch can be skipped without any impact on the rest 
      of the series.
  server: Give up after a while if GStreamer cannot handle the video
  server: Respect the GStreamer encoder's valid bit rate range
        set_gstenc_bitrate() should now pass the right integer type to 
        g_object_set().
  server: Add support for GStreamer 0.10
      Reduced the number of #ifdefs for GStreamer 0.10 using 
      Christophe's suggestions.
  spiceqxl: Add SpiceVideoCodecs to specify video codec preferences
  Xspice: Add --video-codecs to specify encoder and codec preferences
  spice-gtk: Enable adding alternative video decoders
  spice-gtk: Optimize handling of the decoded frame buffer
  spice-gtk: Rename num_drops_on_receive to arrive_late_count
  spice-gtk: Let the video decoder queue, schedule and drop the frames
  spice-gtk: Add a GStreamer video decoder for MJPEG, VP8 and h264
    - Tweaked some function names in the client's GStreamer decoder to 
      avoid potential name conflicts.
    - Fixed a race condition in the client which would cause it to 
      freeze when hovering madly over the seek barin YouTube videos.
    - As a side effect the new code now schedules the frames rending 
      itself so it can adapt when the mm-time get yanked around.
    - It also does not queue the frames in the pipeline and thus is less 
      likely to lose them when size changes force us to rebuild it.
  spice-gtk: Allow disabling support for the builtin MJPEG video decoder
  spice-gtk: Probe GStreamer before advertising support for a codec
  spice-gtk: Use decodebin as a fallback for the GStreamer video decoder
  spice-gtk: Avoid GAsyncQueue for compatibility with GLib < 2.46
      This patch is optional and avoids the dependency on 
      g_async_queue_push_front() which is missing in GLib < 2.46 
      (e.g. for Debian 8).

Pavel Grunt (1):
  replay: Add an option to change video codec




Changes from v9:
* Moved the codec parameters except the bitrate to the pipeline 
  'gst-launch' string. That string is traced so this makes it easier 
  to see exactly how the codecs were configured.
* Updated how the video:codec preference string is handled to use a 
  GArray.
* Changed the VP8 encoder parameters to improve performance based on 
  the realtime profile. Furthermore it turns out that it is important 
  to get the number of physical cores right, and that's something the 
  encoder does not know how to do itself.
* Instead of waiting until the last millisecond to decode frames, 
  Spice-Gtk now lets the video decoders do the queuing. So in the 
  GStreamer case the GStreamer pipeline now does the queuing and 
  calls Spice-Gtk back when it's time to display the frame.
* That last point means mm-time adjustments don't apply to queued 
  frames. That does not really seem to be an issue but could be 
  modified if necessary.


Changes from v8:
* Rebased on the current source.
* Adjusted the Spice protocol version requirement.


Changes from v7:
* When the client has support for GStreamer video codecs it now
  checks the presence of each supported codec before advertising 
  support for it.


Changes from v6:
* configure.ac uses the new m4 macros to check for GStreamer support
  and the presence of the needed plugins.
* It separates adding checks for the client codec support from
  specifying which codec to use in the server and from adding VP8
  support.


-- 
2.7.0