Regulate speed of decoded frames using presentation timestamp

[Andressio]

Hi all. I am developing an application that consumes h264 streams from two
different models of ip cameras via rtsp. I am using a gstremer pipeline that
looks like this:

/rtspsrc location=camera_rtsp protocols=GST_RTSP_LOWER_TRANS_TCP ! 
rtph264depay ! h264parse ! 
video/x-h264,stream-format=byte-stream,alignment=au ! appsink/

Buffers pulled from appsink hold encoded h264 packets. I feed these packets
directly into the NVDEC decoder of an nvidia gpu to obtain the decoded raw
frames using opencv + nvidia video codec sdk (note that I don't use the
nvdec element from gstreamer plugins bad).

What happens is that with one of the cameras everything works fine: the
frames are decoded at the right speed (in line with video framerate) and
displaying them results in a smooth video. On the other hand the second
camera gives poor results: frames are decoded at variable speed leading to a
bulky video when displayed but, if I average the times between all
consecutives frames from one keyframe to the next keyframe, the speed is in
line with video framerate. To sum up this last case: all frames are decoded
correctly, but are not decoded at a constant speed.

I am quite confident that the decoder works properly. I guess it is simply a
matter of correctly using the presentation timestamp that can be extracted
from appsink when pulling buffers (N.B. for the first camera the
presentation timestamp is mostly monotonically increasing but sometimes
returns back in time, for the second camera it is monotonically incresing).

In an early stage of the application I put gstreamer's nvdec element between
/h264parse/ end /appsink/. I obtained regular streams with raw frames
emitted at a regular speed for both cameras but unfortunately it was not
very reliable.

How does gstreamer's nvdec regulate the emission of decoded frames? How can
I implement a similar behavior  using the presentation timestamps from the
/appsink/ while keeping the above pipeline unchanged? I guess I should use
some buffers, but are you aware of some examples or resources that can be
used as a staring point?

Thanks




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/