Playback of my RTSP streamed video choppy

Thu Apr 9 16:22:44 PDT 2015

Thanks Tim,

We have tried not setting timestamps and specifying do-timestamp=true with various permutations of pipelines, but so far have found setting the timestamps to provide the best playback.

I did instrument GST_DEBUG=basesink:6 and see that it is not tossing packets.

I then went to wireshark and came up with some very interesting results.

First of all, I was streaming video at 10FPS, with a GOP of 10.  I see 9 RTP packets with timestamps that increment by 9000 (as expected), but the 10th packet timestamp increments by 18000. The is consistent and systematic.  We are dropping a rtp time slot every 9 frames.  From a real time perspective, I see the packets coming out with close to 100 ms intervals, but generally lagging by 20-50ms, and a significant percentage of time gaps as much as 700ms.  The early delivery is overwhelmed by late, so the pipeline starves, and VLC complains about late packets.

I then went to 5FPS with a GOP of 1.  Wireshark is shows consistent increments of 18000 (again as expected) between frames.  But the real time spacing between frames ranges from 120ms-600ms.  It is bursty and what concerns me is most frames are emitted 220ms apart, which I see reflected in VLC with its constant complaining that packets are too late to display or might be too late to display.

I do not believe that pipeline starvation from the appsrc is the causation of the long delays in packetizing because I am capturing the video stream from the camera into a file, and reading the file contents into the appsrc so I can eliminate the jitter that might be caused by collecting video packets over a wireless network.

So I am back to focusing on  
1) Why would the rtph264pay element inconsistently put rtp timestamps, missing a frame slot systematically
2) How can I get this process speed up so it moves the packets to the network in a more timely manner.

-----Original Message-----
From: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of Tim Müller
Sent: Thursday, April 09, 2015 3:44 AM
To: Discussion of the development of and with GStreamer
Subject: Re: Playback of my RTSP streamed video choppy

On Wed, 2015-04-08 at 22:30 +0000, Kerry Calvert wrote:

Hi Kerry,

> My pipeline is as follows: appsrc is-live=true name=h264appsrc
> blocksize=8192 ! h264parse ! rtph264pay name=pay0 pt=96
> config-interval=5 perfect-rtptime=false
> 
> My code putting data into the app src is scanning the input video 
> stream and breaking it into NAL units.  Each NAL unit is then put into 
> a GSTBUFFER, and the PTS timestamp assigned to the GSTBUFFER is the 
> timestamp of the previous GSTBUFFER + incremental time per frame rate.
> So for 5FPS, the timestamps would go 0, 200ms, 400ms, 600ms,800ms, 
> etc.  This algorithm gives us the ability to clearly organize the 
> video data into frames to be played back at a given framerate.

So you are setting the timestamps on the buffers you feed into the pipeline yourself. Thing is, this is a live pipeline, so this might be a bit problematic.

Two things to try:

a) don't set timestamps yourself and use

  appsrc is-live=true do-timestamp=true min-latency=500000000

just to see if it makes any difference.

b) add a queue after appsrc (then you can feed data ahead of time; you may still want to set some min-latency though, to avoid things being dropped at the beginning.)

In a live pipeline the data captured at the source should be timestamped with the current running time of the pipeline clock, and there would typically also be some >0 latency. I suspect what's happening in your case is that you "under-timestamp" your data, so the sink ends up discarding a lot of buffers because it considers them too late. You can figure out if that's what's happening by looking at the
GST_DEBUG=basesink:6 log of the sending (multi)udpsink.

http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-latency.txt

> I feel like something is inefficient in GStreamer’s rtp packetizing or 
> in its ability to push packets onto the network at the desired timing.
> I have the same problem whether I run my camera at 10fps or 5fps.
> 

If it also happens at 5fps it's probably not a matter of efficiency, but something systematically wrong, like the timestamping + latency configuration of the pipeline.

> I am looking for some hints for how to determine where this bottleneck 
> is.  I need to determine if the problem is indeed in GStreamer or is 
> in the network layer.

Looking at the basesink logs should tell you if it drops buffers before sending them. A wireshark capture of the data sent (on the sender side and the receiver side) should also tell you if all data is sent/received or not (by looking at the rtp sequence number and timestamps etc.).

Cheers
 -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel