<div dir="ltr">Hi<br><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 13, 2015 at 10:27 PM, Francois Gouget <span dir="ltr"><<a href="mailto:fgouget@codeweavers.com" target="_blank">fgouget@codeweavers.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The GStreamer video encoder is quite primitive and mostly meant to<br>
introduce, test and debug the basic infrastructure.<br>
The main limitation is the lack of rate control: the bitrate is set at<br>
startup and will not react to changes in the network conditions. Still<br>
it should work fine on LANs.<br>
---<br>
<a href="http://configure.ac" target="_blank">configure.ac</a> | 18 ++<br>
server/Makefile.am | 8 +<br>
server/gstreamer_encoder.c | 441 +++++++++++++++++++++++++++++++++++++++++++++<br>
server/red_worker.c | 11 +-<br>
server/video_encoder.h | 10 +-<br>
5 files changed, 484 insertions(+), 4 deletions(-)<br>
create mode 100644 server/gstreamer_encoder.c<br>
<br>
diff --git a/<a href="http://configure.ac" target="_blank">configure.ac</a> b/<a href="http://configure.ac" target="_blank">configure.ac</a><br>
index 7a9fc4b..ae11301 100644<br>
--- a/<a href="http://configure.ac" target="_blank">configure.ac</a><br>
+++ b/<a href="http://configure.ac" target="_blank">configure.ac</a><br>
@@ -91,6 +91,15 @@ if test "x$enable_smartcard" = "xyes"; then<br>
AC_DEFINE([USE_SMARTCARD], [1], [Define if supporting smartcard proxying])<br>
fi<br>
<br>
+AC_ARG_ENABLE(gstreamer-0.10,<br>
+[ --enable-gstreamer-0.10 Enable GStreamer 0.10 support],,<br>
+[enable_gstreamer_0_10="yes"])<br>
+AS_IF([test x"$enable_gstreamer_0_10" != "xno"], [enable_gstreamer_0_10="yes"])<br>
+AM_CONDITIONAL(SUPPORT_GSTREAMER_0_10, test "x$enable_gstreamer_0_10" != "xno")<br>
+if test "x$enable_gstreamer_0_10" = "xyes"; then<br>
+ AC_DEFINE([HAVE_GSTREAMER_0_10], [1], [Define if supporting GStreamer 0.10])<br>
+fi<br>
+<br></blockquote><div><br></div><div>I think this should target at least gstreamer 1.0. Do you have good reasons to want 0.10 support?<br> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
AC_ARG_ENABLE(automated_tests,<br>
[ --enable-automated-tests Enable automated tests using spicy-screenshot (part of spice--gtk)],,<br>
[enable_automated_tests="no"])<br>
@@ -127,6 +136,13 @@ if test "x$enable_smartcard" = "xyes"; then<br>
AS_VAR_APPEND([SPICE_REQUIRES], [" libcacard >= 0.1.2"])<br>
fi<br>
<br>
+if test "x$enable_gstreamer_0_10" = "xyes"; then<br>
+ PKG_CHECK_MODULES(GSTREAMER_0_10, [gstreamer-0.10, gstreamer-app-0.10])<br>
+ AC_SUBST(GSTREAMER_0_10_LIBS)<br>
+ AC_SUBST(GSTREAMER_0_10_CFLAGS)<br>
+ AS_VAR_APPEND([SPICE_REQUIRES], [" gstreamer-0.10"])<br>
+fi<br>
+<br>
<br>
PKG_CHECK_MODULES([GLIB2], [glib-2.0 >= 2.22])<br>
AS_VAR_APPEND([SPICE_REQUIRES], [" glib-2.0 >= 2.22"])<br>
@@ -359,6 +375,8 @@ echo "<br>
<br>
Smartcard: ${enable_smartcard}<br>
<br>
+ GStreamer-0.10: ${enable_gstreamer-0.10}<br>
+<br>
SASL support: ${enable_sasl}<br>
<br>
Automated tests: ${enable_automated_tests}<br>
diff --git a/server/Makefile.am b/server/Makefile.am<br>
index 36b6d45..28c4a46 100644<br>
--- a/server/Makefile.am<br>
+++ b/server/Makefile.am<br>
@@ -11,6 +11,7 @@ AM_CPPFLAGS = \<br>
$(SASL_CFLAGS) \<br>
$(SLIRP_CFLAGS) \<br>
$(SMARTCARD_CFLAGS) \<br>
+ $(GSTREAMER_0_10_CFLAGS) \<br>
$(SSL_CFLAGS) \<br>
$(VISIBILITY_HIDDEN_CFLAGS) \<br>
$(WARN_CFLAGS) \<br>
@@ -41,6 +42,7 @@ libspice_server_la_LIBADD = \<br>
$(PIXMAN_LIBS) \<br>
$(SASL_LIBS) \<br>
$(SLIRP_LIBS) \<br>
+ $(GSTREAMER_0_10_LIBS) \<br>
$(SSL_LIBS) \<br>
$(Z_LIBS) \<br>
$(SPICE_NONPKGCONFIG_LIBS) \<br>
@@ -138,6 +140,12 @@ libspice_server_la_SOURCES += \<br>
$(NULL)<br>
endif<br>
<br>
+if SUPPORT_GSTREAMER_0_10<br>
+libspice_server_la_SOURCES += \<br>
+ gstreamer_encoder.c \<br>
+ $(NULL)<br>
+endif<br>
+<br>
EXTRA_DIST = \<br>
glz_encode_match_tmpl.c \<br>
glz_encode_tmpl.c \<br>
diff --git a/server/gstreamer_encoder.c b/server/gstreamer_encoder.c<br>
new file mode 100644<br>
index 0000000..ef71b73<br>
--- /dev/null<br>
+++ b/server/gstreamer_encoder.c<br>
@@ -0,0 +1,441 @@<br>
+/* -*- Mode: C; c-basic-offset: 4; indent-tabs-mode: nil -*- */<br>
+/*<br>
+ Copyright (C) 2015 Jeremy White<br>
+ Copyright (C) 2015 Francois Gouget<br>
+<br>
+ This library is free software; you can redistribute it and/or<br>
+ modify it under the terms of the GNU Lesser General Public<br>
+ License as published by the Free Software Foundation; either<br>
+ version 2.1 of the License, or (at your option) any later version.<br>
+<br>
+ This library is distributed in the hope that it will be useful,<br>
+ but WITHOUT ANY WARRANTY; without even the implied warranty of<br>
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU<br>
+ Lesser General Public License for more details.<br>
+<br>
+ You should have received a copy of the GNU Lesser General Public<br>
+ License along with this library; if not, see <<A HREF="<a href="http://www.gnu.org/licenses/" target="_blank">http://www.gnu.org/licenses/</a>"><a href="http://www.gnu.org/licenses/" target="_blank">http://www.gnu.org/licenses/</a></A>>.<br>
+*/<br>
+#ifdef HAVE_CONFIG_H<br>
+#include <config.h><br>
+#endif<br>
+<br>
+#include <gst/gst.h><br>
+#include <gst/app/gstappsrc.h><br>
+#include <gst/app/gstappsink.h><br>
+<br>
+#include "red_common.h"<br>
+<br>
+typedef struct GstEncoder GstEncoder;<br>
+#define VIDEO_ENCODER_T GstEncoder<br>
+#include "video_encoder.h"<br>
+<br>
+typedef struct {<br>
+ SpiceBitmapFmt spice_format;<br>
+ int bpp;<br>
+ int depth;<br>
+ int endianness;<br>
+ int blue_mask;<br>
+ int green_mask;<br>
+ int red_mask;<br>
+} SpiceFormatForGStreamer;<br>
+<br>
+struct GstEncoder {<br>
+ VideoEncoder base;<br>
+<br>
+ /* The GStreamer pipeline. If pipeline is NULL then the other pointers are<br>
+ * invalid.<br>
+ */<br>
+ GstElement *pipeline;<br>
+ GstCaps *src_caps;<br>
+ GstAppSrc *appsrc;<br>
+ GstElement *gstenc;<br>
+ GstAppSink *appsink;<br>
+<br>
+ /* The frame counter for GStreamer buffers */<br>
+ int frame;<br>
+<br>
+ /* Source video information */<br>
+ int width;<br>
+ int height;<br>
+ SpiceFormatForGStreamer *format;<br>
+ SpiceBitmapFmt spice_format;<br>
+<br>
+ /* Rate control information */<br>
+ uint64_t bit_rate;<br>
+<br>
+ /* Rate control callbacks */<br>
+ VideoEncoderRateControlCbs cbs;<br>
+ void *cbs_opaque;<br>
+<br>
+ /* stats */<br>
+ uint64_t starting_bit_rate;<br>
+};<br>
+<br>
+/* The source fps changes all the time so don't store it */<br>
+uint32_t get_source_fps(GstEncoder *encoder)<br>
+{<br>
+ return encoder->cbs.get_roundtrip_ms ?<br>
+ encoder->cbs.get_source_fps(encoder->cbs_opaque) : 25;<br>
+}<br>
+<br>
+static SpiceFormatForGStreamer *map_format(SpiceBitmapFmt format)<br>
+{<br>
+ int i;<br>
+ static SpiceFormatForGStreamer format_map[] = {<br>
+ { SPICE_BITMAP_FMT_RGBA, 32, 24, 4321, 0xff000000, 0xff0000, 0xff00},<br>
+ /* TODO: Check the other formats */<br>
+ { SPICE_BITMAP_FMT_32BIT, 32, 24, 4321, 0xff000000, 0xff0000, 0xff00},<br>
+ { SPICE_BITMAP_FMT_24BIT, 24, 24, 4321, 0xff0000, 0xff00, 0xff},<br>
+ { SPICE_BITMAP_FMT_16BIT, 16, 16, 4321, 0x001f, 0x03E0, 0x7C00},<br>
+ };<br>
+<br>
+ for (i = 0; i < sizeof(format_map) / sizeof(format_map[0]); i++)<br>
+ if (format_map[i].spice_format == format)<br>
+ return &format_map[i];<br>
+<br>
+ return NULL;<br>
+}<br>
+<br>
+static void reset_pipeline(GstEncoder *encoder)<br>
+{<br>
+ if (!encoder->pipeline)<br>
+ return;<br>
+<br>
+ gst_element_set_state(encoder->pipeline, GST_STATE_NULL);<br>
+ gst_caps_unref(encoder->src_caps);<br>
+ gst_object_unref(encoder->appsrc);<br>
+ gst_object_unref(encoder->gstenc);<br>
+ gst_object_unref(encoder->appsink);<br>
+ gst_object_unref(encoder->pipeline); <br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
+ encoder->pipeline = NULL;<br>
+}<br>
+<br>
+static void adjust_bit_rate(GstEncoder *encoder)<br>
+{<br>
+ uint64_t raw_bit_rate;<br>
+ uint32_t fps, compression;<br>
+<br>
+ fps = get_source_fps(encoder);<br>
+ raw_bit_rate = encoder->width * encoder->height * encoder->format->depth * fps;<br>
+<br>
+ if (!encoder->bit_rate)<br>
+ {<br>
+ /* If no bit rate is available start with 10 Mbps and let the rate<br>
+ * control mechanisms trim it down.<br>
+ */<br>
+ encoder->bit_rate = 10 * 1024 * 1024;<br>
+ spice_debug("resetting the bit rate to 10Mbps");<br>
+ }<br>
+ else if (encoder->bit_rate < 20 * 1024)<br>
+ {<br>
+ /* Don't let the bit rate go below 20kbps. */<br>
+ encoder->bit_rate = 20 * 1024;<br>
+ spice_debug("bottoming the bit rate to 20kpbs");<br>
+ }<br>
+<br>
+ compression = raw_bit_rate / encoder->bit_rate;<br>
+ if (compression < 10)<br>
+ {<br>
+ /* Even MJPEG should achieve good quality with a 10x compression level */<br>
+ encoder->bit_rate = raw_bit_rate / 10;<br>
+ spice_debug("capping the bit rate to %.2fMbps for a 10x compression level", ((double)encoder->bit_rate) / 1024 / 1024);<br>
+ }<br>
+ else<br>
+ {<br>
+ spice_debug("setting the bit rate to %.2fMbps for a %dx compression level", ((double)encoder->bit_rate) / 1024 / 1024, compression);<br>
+ }<br>
+}<br>
+<br>
+static int set_appsrc_caps(GstEncoder *encoder)<br>
+{<br>
+ GstCaps *new_caps = gst_caps_new_simple("video/x-raw-rgb",<br>
+ "bpp", G_TYPE_INT, encoder->format->bpp,<br>
+ "depth", G_TYPE_INT, encoder->format->depth,<br>
+ "width", G_TYPE_INT, encoder->width,<br>
+ "height", G_TYPE_INT, encoder->height,<br>
+ "endianness", G_TYPE_INT, encoder->format->endianness,<br>
+ "red_mask", G_TYPE_INT, encoder->format->red_mask,<br>
+ "green_mask", G_TYPE_INT, encoder->format->green_mask,<br>
+ "blue_mask", G_TYPE_INT, encoder->format->blue_mask,<br>
+ "framerate", GST_TYPE_FRACTION, get_source_fps(encoder), 1,<br>
+ NULL);<br>
+ if (!new_caps)<br>
+ {<br>
+ spice_warning("GStreamer error: could not create the source caps");<br>
+ reset_pipeline(encoder);<br>
+ return FALSE;<br>
+ }<br>
+ g_object_set(G_OBJECT(encoder->appsrc), "caps", new_caps, NULL);<br>
+ if (encoder->src_caps)<br>
+ gst_caps_unref(encoder->src_caps);<br>
+ encoder->src_caps = new_caps;<br>
+ return TRUE;<br>
+}<br>
+<br>
+static int construct_pipeline(GstEncoder *encoder, const SpiceBitmap *bitmap)<br>
+{<br>
+ GstStateChangeReturn ret;<br>
+ GError *err;<br>
+<br>
+ err = NULL;<br>
+ encoder->pipeline = gst_parse_launch_full("appsrc name=src is-live=1 ! ffmpegcolorspace ! ffenc_mjpeg name=encoder ! appsink name=sink", NULL, GST_PARSE_FLAG_FATAL_ERRORS, &err);<br></blockquote><div><br></div><div>Any reason to pick ffmpegcolorspace instead of autoconvert ?<br><br></div><div>I would rather see an encodebin with a mjpeg profile or caps. However, I am not sure it will be possible to adjust bitrate easily. That could be done later.<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
+ if (!encoder->pipeline)<br>
+ {<br>
+ spice_warning("GStreamer error: %s", err->message);<br>
+ g_clear_error(&err);<br>
+ return FALSE;<br>
+ }<br>
+ encoder->appsrc = GST_APP_SRC(gst_bin_get_by_name(GST_BIN(encoder->pipeline), "src"));<br>
+ encoder->gstenc = gst_bin_get_by_name(GST_BIN(encoder->pipeline), "encoder");<br>
+ encoder->appsink = GST_APP_SINK(gst_bin_get_by_name(GST_BIN(encoder->pipeline), "sink"));<br>
+<br>
+ /* Set the source caps */<br>
+ encoder->src_caps = NULL;<br>
+ if (!set_appsrc_caps(encoder))<br>
+ return FALSE;<br>
+<br>
+ /* Set the encoder initial bit rate */<br>
+ adjust_bit_rate(encoder);<br>
+ g_object_set(G_OBJECT(encoder->gstenc), "bitrate", encoder->bit_rate, NULL);<br>
+<br>
+ /* Start playing */<br>
+ gst_pipeline_use_clock(GST_PIPELINE(encoder->pipeline), NULL);<br>
+ ret = gst_element_set_state(encoder->pipeline, GST_STATE_PLAYING);<br>
+ if (ret == GST_STATE_CHANGE_FAILURE) {<br>
+ spice_warning("GStreamer error: unable to set the pipeline to the playing state");<br>
+ reset_pipeline(encoder);<br>
+ return FALSE;<br>
+ }<br>
+ return TRUE;<br>
+}<br>
+<br>
+static int reconfigure_pipeline(GstEncoder *encoder)<br>
+{<br>
+ if (gst_element_set_state(encoder->pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE ||<br>
+ !set_appsrc_caps(encoder) ||<br>
+ gst_element_set_state(encoder->pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {<br>
+ spice_warning("GStreamer error: the pipeline reconfiguration failed");<br>
+ reset_pipeline(encoder);<br>
+ return FALSE;<br>
+ }<br>
+ return TRUE;<br>
+}<br>
+<br>
+static inline uint8_t *get_image_line(SpiceChunks *chunks, size_t *offset,<br>
+ int *chunk_nr, int stride)<br>
+{<br>
+ uint8_t *ret;<br>
+ SpiceChunk *chunk;<br>
+<br>
+ chunk = &chunks->chunk[*chunk_nr];<br>
+<br>
+ if (*offset == chunk->len) {<br>
+ if (*chunk_nr == chunks->num_chunks - 1) {<br>
+ return NULL; /* Last chunk */<br>
+ }<br>
+ *offset = 0;<br>
+ (*chunk_nr)++;<br>
+ chunk = &chunks->chunk[*chunk_nr];<br>
+ }<br>
+<br>
+ if (chunk->len - *offset < stride) {<br>
+ spice_warning("bad chunk alignment");<br>
+ return NULL;<br>
+ }<br>
+ ret = chunk->data + *offset;<br>
+ *offset += stride;<br>
+ return ret;<br>
+}<br>
+<br>
+<br>
+static int push_raw_frame(GstEncoder *encoder, const SpiceBitmap *bitmap,<br>
+ const SpiceRect *src, int top_down)<br>
+{<br>
+ SpiceChunks *chunks;<br>
+ uint32_t image_stride;<br>
+ size_t offset;<br>
+ int i, chunk;<br>
+ uint8_t *p;<br>
+<br>
+ GstBuffer *buffer;<br>
+ GstFlowReturn ret;<br>
+<br>
+ const unsigned int stream_height = src->bottom - src->top;<br>
+ const unsigned int stream_width = src->right - src->left;<br>
+<br>
+ /* We could create a bunch of GstMemory objects, one per line, to avoid<br>
+ * copying the raw frame. But this may run into alignment problems and it's<br>
+ * unclear that it would be any faster, particularly if we're unable to<br>
+ * cache these objects.<br>
+ */<br></blockquote><div><br></div><div>It would probably help when encoding fullscreen.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
+ buffer = gst_buffer_new_and_alloc (stream_width * stream_height * (encoder->format->bpp / 8));<br>
+<br>
+ chunks = bitmap->data;<br>
+ offset = 0;<br>
+ chunk = 0;<br>
+ image_stride = bitmap->stride;<br>
+<br>
+ const int skip_lines = top_down ? src->top : bitmap->y - (src->bottom - 0);<br>
+ for (i = 0; i < skip_lines; i++) {<br>
+ get_image_line(chunks, &offset, &chunk, image_stride);<br>
+ }<br>
+<br>
+ for (i = 0, p = GST_BUFFER_DATA(buffer); i < stream_height; i++) {<br>
+ uint8_t *src_line =<br>
+ (uint8_t *)get_image_line(chunks, &offset, &chunk, image_stride);<br>
+<br>
+ if (!src_line) {<br>
+ return VIDEO_ENCODER_FRAME_UNSUPPORTED;<br>
+ }<br>
+<br>
+ src_line += src->left * (encoder->format->bpp / 8);<br>
+<br>
+ memcpy(p, src_line, stream_width * (encoder->format->bpp / 8));<br>
+ p += stream_width * (encoder->format->bpp / 8);<br>
+ }<br>
+<br>
+ /* The GStreamer buffer timestamps and framerate are irrelevant and would<br>
+ * be hard to set right because they can arrive a bit irregularly<br>
+ */<br>
+ GST_BUFFER_TIMESTAMP(buffer) = GST_CLOCK_TIME_NONE;<br>
+ GST_BUFFER_DURATION(buffer) = GST_CLOCK_TIME_NONE;<br>
+ GST_BUFFER_OFFSET(buffer) = encoder->frame++;<br>
+ gst_buffer_set_caps(buffer, encoder->src_caps);<br>
+<br>
+ ret = gst_app_src_push_buffer(encoder->appsrc, buffer);<br>
+ if (ret != GST_FLOW_OK)<br>
+ {<br>
+ spice_debug("unable to push source buffer");<br>
+ return VIDEO_ENCODER_FRAME_UNSUPPORTED;<br>
+ }<br>
+<br>
+ return VIDEO_ENCODER_FRAME_ENCODE_DONE;<br>
+}<br>
+<br>
+static int pull_compressed_buffer(GstEncoder *encoder,<br>
+ uint8_t **outbuf, size_t *outbuf_size,<br>
+ int *data_size)<br>
+{<br>
+ GstBuffer *buffer;<br>
+<br>
+ buffer = gst_app_sink_pull_buffer(encoder->appsink);<br>
+<br>
+ if (buffer) {<br>
+ int len = GST_BUFFER_SIZE(buffer);<br>
+ spice_assert(outbuf && outbuf_size);<br>
+ if (!*outbuf || *outbuf_size < len)<br>
+ {<br>
+ *outbuf = spice_realloc(*outbuf, len);<br>
+ *outbuf_size = len;<br>
+ }<br>
+ memcpy(*outbuf, GST_BUFFER_DATA(buffer), len);<br>
+ gst_buffer_unref(buffer);<br></blockquote><div><br></div><div>Although it is much less important than the memcpy on the src, I could imagine the memcpy could be removed eventually, perhaps a FIXME is worth here.<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
+ *data_size = len;<br>
+ return VIDEO_ENCODER_FRAME_ENCODE_DONE;<br>
+ }<br>
+ return VIDEO_ENCODER_FRAME_UNSUPPORTED;<br>
+}<br>
+<br>
+static void gst_encoder_destroy(GstEncoder *encoder)<br>
+{<br>
+ reset_pipeline(encoder);<br>
+ free(encoder);<br>
+}<br>
+<br>
+static int gst_encoder_encode_frame(GstEncoder *encoder,<br>
+ const SpiceBitmap *bitmap,<br>
+ int width, int height,<br>
+ const SpiceRect *src, int top_down,<br>
+ uint32_t frame_mm_time,<br>
+ uint8_t **outbuf, size_t *outbuf_size,<br>
+ int *data_size)<br>
+{<br>
+ int rc;<br>
+<br>
+ if (!encoder->pipeline || width != encoder->width ||<br>
+ height != encoder->height || encoder->spice_format != bitmap->format) {<br>
+ spice_debug("video format change: width %d -> %d, height %d -> %d, format %d -> %d", encoder->width, width, encoder->height, height, encoder->spice_format, bitmap->format);<br>
+ encoder->format = map_format(bitmap->format);<br>
+ if (!encoder->format) {<br>
+ spice_debug("unable to map format type %d", bitmap->format);<br>
+ return VIDEO_ENCODER_FRAME_UNSUPPORTED;<br>
+ }<br>
+ encoder->spice_format = bitmap->format;<br>
+ encoder->width = width;<br>
+ encoder->height = height;<br>
+ if (encoder->pipeline && !reconfigure_pipeline(encoder))<br>
+ return VIDEO_ENCODER_FRAME_UNSUPPORTED;<br>
+ }<br>
+ if (!encoder->pipeline && !construct_pipeline(encoder, bitmap))<br>
+ return VIDEO_ENCODER_FRAME_DROP;<br>
+<br>
+ rc = push_raw_frame(encoder, bitmap, src, top_down);<br>
+ if (rc == VIDEO_ENCODER_FRAME_ENCODE_DONE)<br>
+ rc = pull_compressed_buffer(encoder, outbuf, outbuf_size, data_size);<br>
+ return rc;<br>
+}<br>
+<br>
+void gst_encoder_client_stream_report(GstEncoder *encoder,<br>
+ uint32_t num_frames, uint32_t num_drops,<br>
+ uint32_t start_frame_mm_time,<br>
+ uint32_t end_frame_mm_time,<br>
+ int32_t end_frame_delay,<br>
+ uint32_t audio_delay)<br>
+{<br>
+ spice_debug("client report: #frames %u, #drops %d, duration %u video-delay %d audio-delay %u",<br>
+ num_frames, num_drops,<br>
+ end_frame_mm_time - start_frame_mm_time,<br>
+ end_frame_delay, audio_delay);<br>
+}<br>
+<br>
+void gst_encoder_notify_server_frame_drop(GstEncoder *encoder)<br>
+{<br>
+ spice_debug("server frame drop");<br>
+}<br>
+<br>
+uint64_t gst_encoder_get_bit_rate(GstEncoder *encoder)<br>
+{<br>
+ return encoder->bit_rate;<br>
+}<br>
+<br>
+void gst_encoder_get_stats(GstEncoder *encoder, VideoEncoderStats *stats)<br>
+{<br>
+ uint64_t raw_bit_rate = encoder->width * encoder->height * encoder->format->bpp * get_source_fps(encoder);<br>
+<br>
+ spice_assert(encoder != NULL && stats != NULL);<br>
+ stats->starting_bit_rate = encoder->starting_bit_rate;<br>
+ stats->cur_bit_rate = encoder->bit_rate;<br>
+<br>
+ /* Use the compression level as a proxy for the quality */<br>
+ stats->avg_quality = 100.0 - raw_bit_rate / encoder->bit_rate;<br>
+ if (stats->avg_quality < 0)<br>
+ stats->avg_quality = 0;<br>
+}<br>
+<br>
+GstEncoder *create_gstreamer_encoder(uint64_t starting_bit_rate, VideoEncoderRateControlCbs *cbs, void *cbs_opaque)<br>
+{<br>
+ GstEncoder *encoder;<br>
+<br>
+ spice_assert(!cbs || (cbs && cbs->get_roundtrip_ms && cbs->get_source_fps));<br>
+<br>
+ gst_init(NULL, NULL);<br>
+<br>
+ encoder = spice_new0(GstEncoder, 1);<br>
+ encoder->base.destroy = &gst_encoder_destroy;<br>
+ encoder->base.encode_frame = &gst_encoder_encode_frame;<br>
+ encoder->base.client_stream_report = &gst_encoder_client_stream_report;<br>
+ encoder->base.notify_server_frame_drop = &gst_encoder_notify_server_frame_drop;<br>
+ encoder->base.get_bit_rate = &gst_encoder_get_bit_rate;<br>
+ encoder->base.get_stats = &gst_encoder_get_stats;<br>
+ encoder->pipeline = NULL;<br>
+<br>
+ if (cbs)<br>
+ encoder->cbs = *cbs;<br>
+ else<br>
+ encoder->cbs.get_roundtrip_ms = NULL;<br>
+ encoder->cbs_opaque = cbs_opaque;<br>
+ encoder->bit_rate = encoder->starting_bit_rate = starting_bit_rate;<br>
+<br>
+ return encoder;<br>
+}<br>
diff --git a/server/red_worker.c b/server/red_worker.c<br>
index c6b39cb..19e27c5 100644<br>
--- a/server/red_worker.c<br>
+++ b/server/red_worker.c<br>
@@ -3077,6 +3077,7 @@ static void red_stream_update_client_playback_latency(void *opaque, uint32_t del<br>
static void red_display_create_stream(DisplayChannelClient *dcc, Stream *stream)<br>
{<br>
StreamAgent *agent = &dcc->stream_agents[get_stream_id(dcc->common.worker, stream)];<br>
+ create_video_encoder_proc create_video_encoder;<br>
<br>
stream->refs++;<br>
spice_assert(region_is_empty(&agent->vis_region));<br>
@@ -3091,6 +3092,12 @@ static void red_display_create_stream(DisplayChannelClient *dcc, Stream *stream)<br>
agent->fps = MAX_FPS;<br>
agent->dcc = dcc;<br>
<br>
+#ifdef HAVE_GSTREAMER_0_10<br>
+ create_video_encoder = &create_gstreamer_encoder;<br>
+#else<br>
+ create_video_encoder = &create_mjpeg_encoder;<br>
+#endif<br>
+<br>
if (dcc->use_video_encoder_rate_control) {<br>
VideoEncoderRateControlCbs video_cbs;<br>
uint64_t initial_bit_rate;<br>
@@ -3100,9 +3107,9 @@ static void red_display_create_stream(DisplayChannelClient *dcc, Stream *stream)<br>
video_cbs.update_client_playback_delay = red_stream_update_client_playback_latency;<br>
<br>
initial_bit_rate = red_stream_get_initial_bit_rate(dcc, stream);<br>
- agent->video_encoder = create_mjpeg_encoder(initial_bit_rate, &video_cbs, agent);<br>
+ agent->video_encoder = create_video_encoder(initial_bit_rate, &video_cbs, agent);<br>
} else {<br>
- agent->video_encoder = create_mjpeg_encoder(0, NULL, NULL);<br>
+ agent->video_encoder = create_video_encoder(0, NULL, NULL);<br>
}<br>
red_channel_client_pipe_add(&dcc->common.base, &agent->create_item);<br>
<br>
diff --git a/server/video_encoder.h b/server/video_encoder.h<br>
index 8a9f9c6..d5f7fb8 100644<br>
--- a/server/video_encoder.h<br>
+++ b/server/video_encoder.h<br>
@@ -48,8 +48,8 @@ struct VideoEncoder {<br>
*<br>
* @encoder: The video encoder.<br>
* @bitmap: The Spice screen.<br>
- * @width: The width of the Spice screen.<br>
- * @height: The heigth of the Spice screen.<br>
+ * @width: The width of the Spice screen. FIXME: Wrong?<br>
+ * @height: The heigth of the Spice screen. FIXME: Wrong?<br></blockquote><div><br></div><div>The video encoder is usually created for a smaller portion than screen size, for the detected video region. However, since the CAP_SIZED_STREAM, the video region may change for every frame..<br> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* @src: A rectangle specifying the area occupied by the video.<br>
* @top_down: If true the first video line is specified by src.top.<br>
* @outbuf: The buffer for the compressed frame. This must either be<br>
@@ -155,8 +155,14 @@ typedef struct VideoEncoderRateControlCbs {<br>
* @return: A pointer to a structure implementing the VideoEncoder<br>
* methods.<br>
*/<br>
+typedef VIDEO_ENCODER_T* (*create_video_encoder_proc)(uint64_t starting_bit_rate, VideoEncoderRateControlCbs *cbs, void *cbs_opaque);<br>
+<br>
VIDEO_ENCODER_T* create_mjpeg_encoder(uint64_t starting_bit_rate,<br>
VideoEncoderRateControlCbs *cbs,<br>
void *cbs_opaque);<br>
<br>
+VIDEO_ENCODER_T* create_gstreamer_encoder(uint64_t starting_bit_rate,<br>
+ VideoEncoderRateControlCbs *cbs,<br>
+ void *cbs_opaque);<br>
+<br>
#endif<br>
<span><font color="#888888">--<br>
2.1.4<br>
<br>
_______________________________________________<br>
Spice-devel mailing list<br>
<a href="mailto:Spice-devel@lists.freedesktop.org" target="_blank">Spice-devel@lists.freedesktop.org</a><br>
<a href="http://lists.freedesktop.org/mailman/listinfo/spice-devel" target="_blank">http://lists.freedesktop.org/mailman/listinfo/spice-devel</a><br>
</font></span></blockquote></div><br></div><div class="gmail_extra">overall, it looks good to me<br clear="all"></div><div class="gmail_extra"><br>-- <br><div>Marc-André Lureau</div>
</div></div></div>