<div dir="ltr"><div>Thanks! I like a simple fix.... autovideosink sync=false fixed it!</div><div><br></div><div>FYI I had tried framerate 0/1 - I think this is the default if you don't set anything manually<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 22 Sep 2020 at 12:27, Gary Metalle <<a href="mailto:Gary.Metalle@rvl.co.uk">Gary.Metalle@rvl.co.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr" style="font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255);font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi<br>
</p>
<p><br>
What happens if you use a framerate of 0/1? This is the way to define a variable framerate.<br>
</p>
<p><br>
</p>
<p>Maybe the issue is with your decoder pipeline and the not the encoding. Have you tried setting 'sync=FALSE' for the video sink you're using?<br>
</p>
<div style="color:rgb(33,33,33)">
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-3849939215528772395divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> gstreamer-devel <<a href="mailto:gstreamer-devel-bounces@lists.freedesktop.org" target="_blank">gstreamer-devel-bounces@lists.freedesktop.org</a>> on behalf of David J <<a href="mailto:davywj@gmail.com" target="_blank">davywj@gmail.com</a>><br>
<b>Sent:</b> 22 September 2020 12:21<br>
<b>To:</b> Discussion of the development of and with GStreamer<br>
<b>Subject:</b> x264enc and variable rate appsrc</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>I have created a test pipeline (see code below) using a custom appsrc that creates an h264 stream and sends over rtp.</div>
<div><br>
</div>
<div>I am testing it with the following command line to receive and play the video stream:</div>
<div>gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink</div>
<div><br>
</div>
<div>The command-line inside the application looks like this:</div>
<div>appsrc name=MyAppSrc ! video/x-raw,width=640,height=360,framerate=10/1 ! videoconvert ! identity name=ident ! x264enc zerolatency=true speed-preset=superfast ! rtph264pay ! udpsink host=127.0.0.1 port=5000</div>
<div><br>
</div>
<div>At the moment the appsrc is generating frames at precisely 10fps and I set the buffer timestamps correctly. However, unless I also specify the framerate (in the caps string following the appsrc) as 10/1 the client viewer constantly complains of:</div>
<div>WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSinkBin:autovideosink0-actual-sink-d3d11video/GstD3D11VideoSink:d3d11videosink0: A lot of buffers are being dropped.<br>
Additional debug info:<br>
../libs/gst/base/gstbasesink.c(3134): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstD3D11VideoSinkBin:autovideosink0-actual-sink-d3d11video/GstD3D11VideoSink:d3d11videosink0:<br>
There may be a timestamping problem, or this computer is too slow.</div>
<div><br>
</div>
<div><br>
</div>
<div><b>The problem is:</b></div>
<div>In the real application, the appsrc will produce frames at a variable rate - between about 2 and 20 fps. How can I get this to work if I'm required to set a fixed framerate?</div>
<div><br>
</div>
<div>Provided the buffer timestamps are set, why does the framerate property have any bearing on the pipeine?</div>
<div><br>
</div>
<div>Here is the application code:</div>
<div><br>
</div>
<div>// NativeBasicPipeline.cpp : This file contains the 'main' function. Program execution begins and ends there.<br>
//<br>
<br>
#include <gst/gst.h><br>
#include <gst/app/gstappsrc.h><br>
<br>
GTimer* timer;<br>
<br>
GstAppSrc* appSrc;<br>
<br>
guint sourceid = 0;<br>
<br>
int width = 1;<br>
<br>
int colour = 255;<br>
<br>
bool read_data()<br>
{<br>
static GstClockTime time = 0;<br>
static guint64 offset = 0;<br>
<br>
GstFlowReturn ret;<br>
auto ms = g_timer_elapsed(timer, NULL);<br>
<br>
if (ms > 1.0 / 10.0) {<br>
auto size = 640 * 3 * 360;<br>
<br>
gpointer copy = g_malloc(size);<br>
<br>
memset(copy, colour % 255, size);<br>
<br>
colour += 10;<br>
<br>
GstBuffer* buffer = gst_buffer_new_wrapped(copy, size);<br>
<br>
<br>
//GST_BUFFER_TIMESTAMP(buffer) = pts;<br>
GST_BUFFER_PTS(buffer) = time;<br>
GST_BUFFER_DTS(buffer) = time;<br>
GST_BUFFER_DURATION(buffer) = 100 * GST_MSECOND;<br>
<br>
GST_BUFFER_OFFSET(buffer) = offset++;<br>
GST_BUFFER_OFFSET_END(buffer) = offset;<br>
<br>
time += 100 * GST_MSECOND;<br>
<br>
g_signal_emit_by_name(appSrc, "push-buffer", buffer, &ret);<br>
<br>
//ret = gst_app_src_push_buffer(appSrc, buffer);<br>
gst_buffer_unref(buffer);<br>
<br>
g_timer_start(timer);<br>
<br>
return TRUE;<br>
}<br>
<br>
return TRUE;<br>
}<br>
<br>
void start_feed(GstElement* pipeline, guint size, void* unused)<br>
{<br>
if (sourceid == 0) {<br>
sourceid = g_idle_add((GSourceFunc)read_data, 0);<br>
}<br>
}<br>
<br>
void stop_feed(GstElement* pipeline, void* unused)<br>
{<br>
if (sourceid != 0) {<br>
g_source_remove(sourceid);<br>
sourceid = 0;<br>
}<br>
}<br>
<br>
static gboolean print_field(GQuark field, const GValue* value, gpointer pfx) {<br>
gchar* str = gst_value_serialize(value);<br>
<br>
g_print("%s %15s: %s\n", (gchar*)pfx, g_quark_to_string(field), str);<br>
g_free(str);<br>
return TRUE;<br>
}<br>
<br>
static void print_caps(const GstCaps* caps, const gchar* pfx) {<br>
guint i;<br>
<br>
g_return_if_fail(caps != NULL);<br>
<br>
if (gst_caps_is_any(caps)) {<br>
g_print("%sANY\n", pfx);<br>
return;<br>
}<br>
if (gst_caps_is_empty(caps)) {<br>
g_print("%sEMPTY\n", pfx);<br>
return;<br>
}<br>
<br>
for (i = 0; i < gst_caps_get_size(caps); i++) {<br>
GstStructure* structure = gst_caps_get_structure(caps, i);<br>
<br>
g_print("%s%s\n", pfx, gst_structure_get_name(structure));<br>
gst_structure_foreach(structure, print_field, (gpointer)pfx);<br>
}<br>
}<br>
<br>
void handoff(GstElement* pipeline, GstBuffer* buffer, void* unused)<br>
{<br>
static int count = 0;<br>
static GstBuffer* buffers[50];<br>
<br>
//auto ident = gst_bin_get_by_name(GST_BIN(pipeline), "ident");<br>
auto pads = GST_ELEMENT_PADS(pipeline);<br>
<br>
auto pad0 = GST_PAD(pads->data);<br>
auto pad1 = GST_PAD(pads->next->data);<br>
<br>
auto caps = gst_pad_get_current_caps(pad1);<br>
<br>
print_caps(caps, "");<br>
<br>
if (count < 50)<br>
{<br>
GstBuffer* copy = gst_buffer_copy(buffer);<br>
<br>
buffers[count] = copy;<br>
<br>
++count;<br>
}<br>
else<br>
{<br>
count = 100;<br>
<br>
<br>
}<br>
}<br>
<br>
int main()<br>
{<br>
GstElement* pipeline;<br>
GstBus* bus;<br>
GstMessage* msg;<br>
<br>
timer = g_timer_new();<br>
<br>
<br>
/* Initialize GStreamer */<br>
gst_init(0, 0);<br>
<br>
/* Build the pipeline */<br>
<br>
GError* err = 0;<br>
<br>
//auto udpPipe = "videotestsrc pattern=smpte ! video/x-raw,width=640,height=360,framerate=10/1 ! videoscale ! videoconvert ! identity name=ident ! x264enc zerolatency=true speed-preset=superfast ! rtph264pay ! udpsink host=127.0.0.1 port=5000";<br>
<br>
//auto videoPipe = "videotestsrc pattern=smpte ! video/x-raw,width=640,height=360,framerate=10/1 ! videoscale ! videoconvert ! autovideosink";<br>
<br>
auto appSrcPipe = "appsrc name=MyAppSrc ! video/x-raw,width=640,height=360 ! videoconvert ! identity name=ident ! x264enc zerolatency=true speed-preset=superfast ! rtph264pay ! udpsink host=127.0.0.1 port=5000";<br>
<br>
//auto appSrcPipeVideo = "appsrc name=MyAppSrc ! videoscale ! videoconvert ! autovideosink";<br>
<br>
pipeline =<br>
gst_parse_launch<br>
(appSrcPipe,<br>
&err);<br>
<br>
appSrc = (GstAppSrc*)gst_bin_get_by_name(GST_BIN(pipeline), "MyAppSrc");<br>
<br>
g_object_set(G_OBJECT(appSrc), "format", GST_FORMAT_TIME, NULL);<br>
<br>
g_signal_connect(appSrc, "need-data", G_CALLBACK(start_feed), 0);<br>
g_signal_connect(appSrc, "enough-data", G_CALLBACK(stop_feed), 0);<br>
<br>
auto caps = gst_caps_new_simple("video/x-raw",<br>
"format", G_TYPE_STRING, "RGB",<br>
"bpp", G_TYPE_INT, 24,<br>
"depth", G_TYPE_INT, 24,<br>
"width", G_TYPE_INT, 640,<br>
"height", G_TYPE_INT, 360,<br>
NULL);<br>
<br>
gst_app_src_set_caps((GstAppSrc*)appSrc, caps);<br>
<br>
//GstPad* pad = gst_element_get_static_pad(appSrc, "src");<br>
<br>
//auto ident = gst_bin_get_by_name(GST_BIN(pipeline), "ident");<br>
<br>
//g_signal_connect(ident, "handoff", G_CALLBACK(handoff), 0);<br>
<br>
<br>
/* Start playing */<br>
gst_element_set_state(pipeline, GST_STATE_PLAYING);<br>
<br>
auto main_loop = g_main_loop_new(NULL, FALSE);<br>
g_main_loop_run(main_loop);<br>
<br>
/* Wait until error or EOS */<br>
bus = gst_element_get_bus(pipeline);<br>
msg =<br>
gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE,<br>
(GstMessageType)(GST_MESSAGE_ERROR | GST_MESSAGE_EOS));<br>
<br>
/* Free resources */<br>
if (msg != NULL)<br>
gst_message_unref(msg);<br>
gst_object_unref(bus);<br>
gst_element_set_state(pipeline, GST_STATE_NULL);<br>
gst_object_unref(pipeline);<br>
return 0;<br>
}<br>
<br>
<br>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
gstreamer-devel mailing list<br>
<a href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank">gstreamer-devel@lists.freedesktop.org</a><br>
<a href="https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel" rel="noreferrer" target="_blank">https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a><br>
</blockquote></div>