appsrc + h264/rtp streaming

[Tim Müller]

On Tue, 2017-08-08 at 15:44 +0000, Martin Vachovski wrote:

Hi Martin,

> Transmitter:
> AppSrcPipeline.pipeline = gst_parse_launch(
> "appsrc name=appsrc_element block=true ! video/x-
> raw,format=RGB,width=320,height=240,framerate=30/1 ! identity check-
> imperfect-timestamp=true ! "
> "videoconvert ! x264enc ! video/x-h264,profile=\"high-4:4:4\" !
> rtph264pay ! udpsink host=192.168.168.98",
> &err);

Later when you are feeding actual live video you may want to set this
to is-live=true and set min-latency=33333333 or such. With your current
code this shouldn't matter though since there you can generate data
faster than real-time.

> /*GST_BUFFER_PTS(buffer) = timestamp;                            //
> TIMESTAMP
> GST_BUFFER_DURATION(buffer) = duration;
> GST_BUFFER_OFFSET(buffer) = offset++;
> GST_BUFFER_OFFSET_END(buffer) = offset;*/

This appears to be commented out? You should set at least PTS and
DURATION.

> So the problem is that only the first frame is ever decoded and
> displayed on the other side.
> The transmitter pipeline is in PLAYING state and also there is
> network traffic, but only one frame is ever drawn.
> I have tried different modifications to the pipeline in order to
> understand which combination of elements works and which doesn't

You can do something like:

gst-launch-1.0 -v ... ! rtpjitterbuffer ! fakesink silent=false

to see if buffers make it through udpsrc + jitterbuffer.

Once that works, add more elements, i.e. rtph264depay.

To see where the problem comes from. Once you have it narrowed down,
look at the GST_DEBUG log for the problematic element.

> I have tried to provide timestamps- if I uncomment
> the "TIMESTAMP" code block I get an error in the logs and the
> pipeline doesn't seem to start at all
> GStreamer-CRITICAL **: gst_segment_to_running_time: assertion
> 'segment->format == format' failed
> This error message seems to be quite generic so googling it didn't
> show up any relevant discussions

Try setting the "format" of appsrc to GST_FORMAT_TIME (3):

g_object_set (appsrc, "format", GST_FORMAT_TIME, NULL);


> Also I have noticed that if I use the combination
> appsrc + autovideosink
> the data_feed function is called about 20-30 times a second (which
> corresponds to the desired framerate)
> but if I use 
> appsrc + h264 encoding + RTP streaming
> The function is called a couple of hundreds of times per second and
> utilizes 100% CPU

By default x264enc will aim for best quality rather than low latency,
which means it may consume a couple of seconds of video before
outputting anything. In this case it will consume data faster than
real-time at the beginning. Eventually it will be throttled by the sink
syncing to the clock. You can use

  x264enc tune=zerolatency

to make it not do that. This reduces quality, you can tweak other
parameters to avoid this too, this is just the most commonly used.

If you use appsrc ! autovideosink it will be throttled from the start.

Cheers
 -Tim


-- 
Tim Müller, Centricular Ltd - http://www.centricular.com

Join us at the GStreamer Conference!
21-22 October 2017 in Prague, Czech Republic
http://gstreamer.freedesktop.org/conference/