[Bug 720104] videorate: Should be able to duplicate frames in a live stream

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=720104
  GStreamer | gst-plugins-base | unspecified

--- Comment #8 from Andreea Fulger <andreea.fulger at parrot.com> 2014-03-24 14:26:28 UTC ---
To be more specific, here is my use case:

I would like to insert black images (or whatever predefined image) on timeout
in case of a signal loss and insert the real frames as soon as we get the
signal back without stopping the pipeline. 

Duplicating frames seemed to be a good way to start as we can duplicate the
last frame until a time threshold is met and then and only then insert the
black / predefined image.

In my pipeline, the videorate element is preceded by a decoder that doesn't
provide the frames in a regular manner, that is the time between the reception
of two consecutive frames is not equal with the frame's duration.

Here is an example:

frame_duration = 40ms

    Reception time

1. 0:00:03.010784018   got buffer with timestamp 0:00:03.124476778
2. 0:00:03.332974336   got buffer with timestamp 0:00:03.164476778
3. 0:00:03.373752503   got buffer with timestamp 0:00:03.204476778
4. 0:00:03.410815013   got buffer with timestamp 0:00:03.284476778
5. 0:00:03.427801185   got buffer with timestamp 0:00:03.324476778

We can see here that the packet no. 2 arrives 320 ms later than the packet no.
1, while the third and fourth packet arrive at a 40ms interval and finally that
packet no. 5 arrives at a 17ms interval from packet no. 4.

If I set a timeout according to the expected frame rate (that is inserting a
frame each 40ms) duplicate frames will have been inserted before receiving the
packet no. 2. If the output frame rate equals the input frame rate, packets no.
2 - 5 will be dropped because their timestamps are already too old. This also
means that we had a regular buffer injection thanks to the timeout mechanism,
followed by at least 100 ms of no buffer injection at all due to dropping
incoming buffers. This doesn't seem to be a correct way of handling live
streams - we've replaced real buffers by duplicate ones without getting round
the lag problem.

Do you see a solution for this use case? What would be a more appropriate
approach?

For having a regular output I guess I could modify the videorate element such
that instead of dropping frames 2 - 5, I could insert them with the next
theoretical timestamps, but this means that I artificially extend the duration
of the input signal and I am not that familiar with Gstreamer to grasp the
impact on the next elements in my pipeline.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.