Bytestream or sample jitterbuffer

[Nicolas Dufresne]

Le jeudi 27 janvier 2022 à 18:05 +0100, Peter Maersk-Moller via gstreamer-devel
a écrit :
> Hi.
> 
> Every now and then I have kind of needed a sort of jitterbuffer for a
> bytestream, but can't really find the right solution for GStreamer. Can
> anybody suggest a solution to this?
> 
> Assuming you have a simple setup (just an example):
> 
>   gst-launch-1.0 fdsrc ! $SOMEFORMAT ! audioparse ! queue  ! audioconert !
> autoaudiosink
> 
> Here is the thing. Because there is no jitterbuffer in the system, then to
> avoid gap in playout, the autoaudiosink MUST receive N+1th audio frame no
> later d time later it received the Nth audio frame and where d=duration of the
> Nth audio frame. What would be needed would be an element that would not
> forward data before it had received N amounts of data or T amount of time data
> since it received the first data.
> 
> Now queue elements have a limit you can set for when it will start forwarding
> data meaning something like .... you must contain at least N amount of data
> before forwarding, but it is not a jitterbuffer as it can not go below the
> threshold. A jitterbuffer needs to be able to do that. The jitterbuffer
> element can only handle RTP data and I find it a little excessive to have to
> encapsulate the audio data in RTP first and then have the jitterbuffer queue
> it up and then decapsulate it again. If the queue element was allowed to drain
> below its threshold, we would in effect have ajitterbuffer working on bytes,
> buffers and time.
> 
> Any suggestions?

Somewhat your pipeline is live, or can be made live. One way to reach this
effect, could be to place clocksync element after your parser (since you need
timeinformation). This will make you pipeline live, and then you can control the
amount of jitter you allow with the processing-deadline properly for the "real
sink", and ts-offset on the clocksync (or something along these line).
> 
> Thanks in advance.
> Peter Maersk-Moller
> 
>