Best way to fix video encoder issues with 0/1 framerate

[Nicolas Dufresne]

Le dimanche 10 mars 2013 à 23:13 +0000, Thomas Greenwood a écrit :
> I think I can see your changes on that bug, but I don't understand the
> effect of the latency on the video encoders.  I may need your patch as
> well to cover some scenarios when RTP streaming.  Perhaps you could
> explain your patch and sink recovery mode.

Basically, in a live pipeline (RTP stream, video camera, broadcasting,
live transcoding, etc) you have no guaranty to receive all the frames
plus you need not to lag behind during playback. The latency becomes
authoritative, in the sense that if a frame arrives to a sink later then
the expected timestamps plus latency, it will be dropped. In this case
sink could be a video sink to display on screen, a udp sink to send over
network, etc. The recovery mode, is a mechanism that display exactly 1
of the late frames every seconds, this avoid not displaying anything for
a long amount of time, I think this only applies to video sink.

The use case I was working on, was the UVC cameras. UVC cameras often
reduce the framerate to compensate low light. This means, the camera may
advertise 30 frames/second, but in fact this is a maximum, and actual
framerate is much lower. My goal was to be able to encode from a UVC
camera (a live stream) with an encoder that has frame based latency
(x264enc). This means that the latency (expressed in time) will vary as
the duration of a frame vary. My attempt was to increase gradually the
latency to match the worst case.

regards,
Nicolas