[gst-devel] [CONCLUSION] ffdec and QoS

[Arnout Vandecappelle]

On Wednesday 10 June 2009 13:53:19 Arnout Vandecappelle wrote:
> On Wednesday 10 June 2009 10:33:07 Wim Taymans wrote:
> > >  Sounds good.  How does ffmpeg know the latency, though?  It buffers
> > > one frame, but since the framerate is unknown you can't convert this to
> > > time, can you?
> >
> > The thing is that the latency query will be done when the sink prerolls,
> > which means that the decoder has at least decoded and pushed 1 buffer.
> > You could use the info you got from decoding this buffer to answer to
> > the latency query.
>
>  Hm, there's no clock yet, so timestamps are -1...
>
>  Regards,
>  Arnout

 In the end, I added a kind of jitterbuffer at the end of the pipeline (much 
like rtspsrc adds an rtpjitterbuffer).  It becomes like this:

uridecodebin ! queue min-threshold-time=1000000000 max-size-buffers=3 ! autovideosink

 (Actually I patched playsink to have the min-threshold-time).

 I did have to patch uridecodebin to set the is-live property (otherwise 
the sink won't use the upstream latency).  I also patched queue so it
considers a full queue to be non-empty - otherwise, (with a framerate < 3fps)
the queue can fill up with 3 buffers and still be considered empty.

 The reason this works is because the queue now sets a minimum latency of 1s,
but the latency it introduces is usually slightly less than 1s (especially if
it pushes buffers out because of the max-size-buffers).

 Note that this ceases to work if the framerate of the live source is lower 
than 5fps.  Indeed, the 4 buffers in-flight (3 in the queue + 1 in ffmpeg)
take almost a full second, so some of the buffers will be late.  At 4fps,
the late buffers are discarded in the sink, while at 2fps, it degenerates
back to the original behaviour of dropping frames in the transform element.


 I wouldn't really call this a solution, but it works for the moment... The
only alternative I see is disabling QoS completely.

 A better solution is probably to make the queue behave a bit more like a
jitter buffer, i.e. let it fill up to some configurable size initially,
but once the threshold is reached start pushing buffers ASAP.  And even
better, it could dynamically grow and shrink the queue size in reaction
to QoS events.

 Regards,
 Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  D206 D44B 5155 DF98 550D  3F2A 2213 88AA A1C7 C933