[Bug 791285] gstaggregator with start-time-selection=first uses incorrect base_time when waiting for data on sink pads

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=791285

--- Comment #19 from Nicolas Dufresne (stormer) <nicolas at ndufresne.ca> ---
(In reply to Sergey Mamonov from comment #16)
> > ...It calculate when is the deadline for data to be
> > produce in order for that data to be on-time at the sink.
> 
> Definitely we define the term 'on time' different way. According
> audiomixer's logic output buffer has to be sent before the moment of time:
> 
> base_time + next_output_buffer_pts + latency,
> where base_time is the absolute time when audiomixer moved its state to
> PLAYING
> It means that as soon audiomixer is started it tries to produce aggregated
> buffers from absolute base_time moment till the current moment of absolute
> time, and it pushes this aggregated buffers as fast as it can. That leads to
> bursting of buffers with silent right after audiomixer receives the first
> buffer. I.e audiomixer produces on its src pad the following buffers:

This burstring will not happen if the data is not late. That's beacause we so
"+ some_latency" which is queried upstream. In RTP case, this is usually
something provided by rtpjitterbuffer (and configurable).

> 
> I assume that 'on time' should be calculated the following way:
> first_incoming_buffer_time + next_output_buffer_pts + latency,
> where first_incoming_buffer_time is absolute time when the first buffer
> appeared on the one of mixer's sink pads.

No, time is always in relation to the clock. The calculation is a bit more
complex then looking at the PTS.

 1) PTS -> running-time (see gst_segment_to_running_time())
 2) OnTime if: running-time + some_latency <= clock_time_now

> > In live mode, we pop data from the internal queues and mix them. If a source
> > is having a huge latency (like in your test, 5s) that is not declared, then
> > that source will be ignored from the mixing. 
> 
> In my test you can see that the first messages which are late are not
> ignored by the mixer, as well as few next messages (~5).
> That's what I tried to explain in item 2
> 
> 
> > ...but does not seem valid. With RTP, if you have this case, you can increase
> > the latency on rtpjitterbuffer element. What is not out-of-the box is this
> > latency, we don't have a dynamic latency mode in GStreamer.
> 
> I expected that rtpjitterbuffer's latency shows what is a maximum time
> rtpjitterbuffer can keep incoming message before it has to be produced. It
> is needed for fix some possible issues with message reordering and bursting
> which can happend during sending messages over UDP.

No, rtpjitterbuffer don't have an upper limit by default unless you set
drop-on-latency property to TRUE. It's kind of a missing feature to have some
proper limits. drop-on-latency is a bit too agressive for most use cases.

The rtpjitterbuffer buffer latency will have an impact on the amount of data
that will be buffered by the jitterbuffer though. It should reflect the
transmission delay plus some acceptable time cause by network jitter (e.g. wifi
burst).

This element is also responsible for setting proper timestamp that correlates
to the pipeline clocks. I'm pretty sure that some of the timestamp mode are
incompatible with the audiomixer (and any code that would like to do live
synch).

> 
> Im not sure I can calculate and set latency for RTP and audiomixer, because
> the moment when pipeline is ready to receive messages is not the moment when
> it receives the first incoming RTP message. 
> Also it is looks bit strange for me that it is not enougth just prepare
> pipeline and wait while it start to receive data.

It's a very difficult problem to implement dynamic jitterbuffer. No one in the
GStreamer community have proposed a solution. To be fair, audiomixer works for
my use case, but I use the default slave mode, not the none mode that you seem
to use.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.