Dear gstream list members,<div><br></div><div>Is there known performance problems with displaying h264 vidoes with variable frame rate on OS X?</div><div>With the same gstreamer pipeline we are experiencing different performance on windows, linux and os x. We are experiencing latency problems using both glimagesink and osxvideosink on OS X. (Following below is a description of the system)</div>
<div><div><br></div><div>Our company is developing a device which is producing a h264 video stream with variable framerate(10-100fps) and fixed bit rate. The requirement is for a low latency system, where the video decode needs to be as fast and with lowest possible latency. </div>
<div><br></div><div>We are using Gstreamer for decode and display the decoded video data on Linux, windows and mac os x.</div><div>The whole application is written in QT, the code base is share for the windows, linux and mac platform. Only OS specific part is the </div>
<div><br></div><div>On Linux and windows there is non noticeable latency for decode and displaying of the image regardless of fps.</div><div><br></div><div>On OSX the gstreamer pipeline instroduces 1-5 seconds of image latency. It seams that the gstreamer pipeline is not able to cope with the variable framerate stream. The effect is more visible with periods with high framerate. (40-80fps) The decoder becomes more and more unsynchronized to the data provided by the video source when displaying periods with high frame rates, when returning to low framerates (less than 20fps) the delay is again back to 1-2 seconds.</div>
<div><br></div><div>The Linux pipeline is a following:</div><div><br></div><div>appsrc->h264parse->ffdec_h264->videoflip->xvimagesink</div><div>(ubuntu 11.04)</div><div><br></div><div>Windows pipeline is following:</div>
<div>appsrc->h264parse->ffdec_h264->ffmpegcolorspace->directdrawsink</div><div>(windows vista)</div><div><br></div><div>Mac osx pipeline is following:</div><div>appsrc->h264parse->ffdec_h264->colorspace->osxvideosink</div>
<div>(OS X 10.6, gstreamer from macports)</div><div>(there is similar problem and performance with the glimagesink)</div><div><br></div><div>Attached is the code for the init and setup of the various streams.</div><div><br clear="all">
Regards<br>Torstein Dybdahl<br>Fxi Technologies AS<br>
<br>
</div><div><br></div><div>---------</div><div><div>#ifdef WIN32</div><div>void SlaveView::initGStreamerAVC() {</div><div> printf ("Init gstreamer for AVC (h264)\n");</div><div><br></div><div> gst_init (NULL, NULL);</div>
<div><br></div><div> pipeline = gst_pipeline_new ("mpeg");</div><div> source = gst_element_factory_make ("appsrc", "app-source");</div><div> demux = gst_element_factory_make ("h264parse", "demuxer");</div>
<div> decoder = gst_element_factory_make ("ffdec_h264", "decoder");</div><div> color = gst_element_factory_make ("ffmpegcolorspace", "color");</div><div> sink = gst_element_factory_make ("directdrawsink", "ddraw");</div>
<div><br></div><div> if (!pipeline || !source || !demux || !decoder || !color || !sink) {</div><div> g_printerr ("An element could not be created. Exiting.\n");</div><div> exit (1);</div><div> }</div><div>
<br></div><div> gst_bin_add_many (GST_BIN (pipeline),</div><div> source, demux, decoder, color, sink, NULL);</div><div><br></div><div> // link elements</div><div> gst_element_link (source, demux);</div>
<div> gst_element_link (demux, decoder);</div><div> gst_element_link (decoder, color);</div><div> gst_element_link (color, sink);</div><div><br></div><div> // element parameters</div><div> g_object_set (G_OBJECT (sink),</div>
<div> "qos", (gboolean)false, </div><div> "sync", (gboolean)false,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>"max-lateness", (gint)-1,</div>
<div> NULL);</div><div><br></div><div> // qt overlay</div><div> gst_x_overlay_set_xwindow_id (GST_X_OVERLAY (sink), GPOINTER_TO_INT(winId()));</div><div><br></div><div> // start pipeline</div><div> gst_element_set_state (pipeline, GST_STATE_PLAYING);</div>
<div>}</div><div>#elif __APPLE__</div><div>void SlaveView::initGStreamerAVC() {</div><div> printf ("Init gstreamer for AVC (h264)\n");</div><div><br></div><div> gst_init (NULL, NULL);</div><div><br></div><div>
pipeline = gst_pipeline_new ("mpeg");</div><div> source = gst_element_factory_make ("appsrc", "app-source");</div><div> demux = gst_element_factory_make ("h264parse", "demuxer");</div>
<div> decoder = gst_element_factory_make ("ffdec_h264", "decoder");</div><div> color = gst_element_factory_make ("colorspace", "color");</div><div> sink = gst_element_factory_make ("osxvideosink", "embed");</div>
<div> if (!pipeline || !source || !demux || !decoder || !color || !sink) {</div><div> g_printerr ("An element could not be created. Exiting.\n");</div><div> exit (1);</div><div> }</div><div><br></div><div>
// element parameters</div><div> g_object_set (G_OBJECT (sink),</div><div> "qos", (gboolean)false, </div><div> "sync", (gboolean)false,</div><div> // "embed", (gboolean)true,</div>
<div> "max-lateness", (gint)-1,</div><div> // "framerate" , (gint) 50, </div><div> NULL);</div><div><br></div><div> g_object_set (G_OBJECT (decoder),</div><div>
"skip-frame", (gint) 0,</div><div> "direct-rendering", (gboolean)false,</div><div> // "framerate", (gint) 25,</div><div> NULL);</div>
<div><br></div><div><br></div><div> // add to pipeline</div><div> gst_bin_add_many (GST_BIN (pipeline),</div><div> source, demux, decoder, color,sink, NULL);</div><div><br></div><div> // link elements</div>
<div> gst_element_link (source, demux);</div><div> gst_element_link (demux, decoder);</div><div> gst_element_link (decoder,color);</div><div> gst_element_link (color, sink);</div><div><br></div><div> // qt overlay</div>
<div> gst_x_overlay_set_xwindow_id (GST_X_OVERLAY (sink), winId());</div><div><br></div><div> // start pipeline</div><div> gst_element_set_state (pipeline, GST_STATE_PLAYING);</div><div>}</div><div>#else</div><div>void SlaveView::initGStreamerAVC() {</div>
<div> printf ("Init gstreamer for AVC (h264)\n");</div><div><br></div><div> gst_init (NULL, NULL);</div><div><br></div><div> pipeline = gst_pipeline_new ("mpeg");</div><div> source = gst_element_factory_make ("appsrc", "app-source");</div>
<div> demux = gst_element_factory_make ("h264parse", "demuxer");</div><div> decoder = gst_element_factory_make ("ffdec_h264", "decoder");</div><div> flip = gst_element_factory_make ("videoflip", "flip");</div>
<div> sink = gst_element_factory_make ("xvimagesink", NULL);</div><div> if (!pipeline || !source || !demux || !decoder || !flip || !sink) {</div><div> g_printerr ("An element could not be created. Exiting.\n");</div>
<div> exit (1);</div><div> }</div><div><br></div><div> // element parameters</div><div> g_object_set (G_OBJECT (sink),</div><div> "qos", (gboolean)false, </div><div> "sync", (gboolean)false,</div>
<div> NULL);</div><div><br></div><div> g_object_set (G_OBJECT (decoder),</div><div> "direct-rendering", (gboolean)false,</div><div> NULL);</div><div><br></div><div> g_object_set (G_OBJECT (flip),</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span>"method", (guint)0,</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>NULL);</div><div><br></div><div> // add to pipeline</div>
<div> gst_bin_add_many (GST_BIN (pipeline),</div><div> source, demux, decoder, flip,sink, NULL);</div><div><br></div><div> // link elements</div><div> gst_element_link (source, demux);</div><div> gst_element_link (demux, decoder);</div>
<div> gst_element_link (decoder,flip);</div><div> gst_element_link (flip, sink);</div><div><br></div><div> // qt overlay</div><div> gst_x_overlay_set_xwindow_id (GST_X_OVERLAY (sink), GPOINTER_TO_INT(winId()));</div><div>
<br></div><div> // start pipeline</div><div> gst_element_set_state (pipeline, GST_STATE_PLAYING);</div><div>}</div><div><br></div><div>#endif</div></div>
</div>