[Bug 791285] gstaggregator with start-time-selection=first uses incorrect base_time when waiting for data on sink pads

Wed Dec 13 14:57:13 UTC 2017

https://bugzilla.gnome.org/show_bug.cgi?id=791285

--- Comment #16 from Sergey Mamonov <samamonov at gmail.com> ---
(In reply to Nicolas Dufresne (stormer) from comment #15)

I'm sorry, my lack of gstreamer competence make it difficult to explain in a
good way

> I'm not sure I understand 1 and 2 here. In live mode, audiomixer no longer
> waits for upstream data.

Most likley I have misundesrtanding here. Could you please describe what kind
of upstream data do you mean? I expected that audiomixer should wait for
downstream buffers. (I have some upstream/downstream picture in my mind
https://gstreamer.freedesktop.org/documentation/design/overview.html#introduction)
And in live mode audiomixer waits for downstream buffers also, but just for
some time.

> ...It calculate when is the deadline for data to be
> produce in order for that data to be on-time at the sink.

Definitely we define the term 'on time' different way. According audiomixer's
logic output buffer has to be sent before the moment of time:

base_time + next_output_buffer_pts + latency,
where base_time is the absolute time when audiomixer moved its state to PLAYING
It means that as soon audiomixer is started it tries to produce aggregated
buffers from absolute base_time moment till the current moment of absolute
time, and it pushes this aggregated buffers as fast as it can. That leads to
bursting of buffers with silent right after audiomixer receives the first
buffer. I.e audiomixer produces on its src pad the following buffers:

buffer 1 pts 0.00 duration 0.02 at real time t (buffer has data from the sink_0
pad)
buffer 2 pts 0.02 duration 0.02 at real time t+0.0001ms(buffer has data from
the sink_0 pad)
buffer 3 pts 0.04 duration 0.02 at real time t+0.0001ms(buffer has data from
the sink_0 pad)
buffer 4 pts 0.06 duration 0.02 at real time t+0.0002ms(buffer has data from
the sink_0 pad)
buffer 5 pts 0.08 duration 0.02 at real time t+0.0002ms(buffer has data from
the sink_0 pad)
buffer 6 pts 0.10 duration 0.02 at real time t+0.0002ms(buffer has data from
the sink_0 pad)
buffer 7 pts 0.12 duration 0.02 at real time t+0.0003ms(buffer has data from
the sink_0 pad)
buffer 8 pts 0.14 duration 0.02 at real time t+0.0003ms(buffer has silence,
because there is no data on sink_0 pad, it has no arrived yet)
buffer 9 pts 0.14 duration 0.02 at real time t+0.0004ms(buffer has silence,
because there is no data on sink_0 pad, it has no arrived yet)
...
(I tried to explain it in my previous message item 1)

I assume that 'on time' should be calculated the following way:
first_incoming_buffer_time + next_output_buffer_pts + latency,
where first_incoming_buffer_time is absolute time when the first buffer
appeared on the one of mixer's sink pads.
I expect that audiomixer will provide aggregated packet each X milliseconds
(where X is about output-buffer-duration property), I assume that audiomixer's
src pad should push buffers the following way:
buffer 1 pts 0.00 duration 0.02 at real time t      (buffer has data from the
sink_0 pad)
buffer 2 pts 0.02 duration 0.02 at real time t+0.02 (buffer has data from the
sink_0 pad, because it waited enougth time ~0.02 ms to receive it)
buffer 3 pts 0.04 duration 0.02 at real time t+0.04 (buffer has data from the
sink_0 pad, because it waited enougth time ~0.02 ms to receive it)
buffer 4 pts 0.06 duration 0.02 at real time t+0.06 (buffer has data from the
sink_0 pad, because it waited enougth time ~0.02 ms to receive it)
buffer 5 pts 0.08 duration 0.02 at real time t+0.08 (buffer has data from the
sink_0 pad, because it waited enougth time ~0.02 ms to receive it)
buffer 6 pts 0.10 duration 0.02 at real time t+0.10 (buffer has data from the
sink_0 pad, because it waited enougth time ~0.02 ms to receive it)
buffer 7 pts 0.12 duration 0.02 at real time t+0.12 (buffer has data from the
sink_0 pad, because it waited enougth time ~0.02 ms to receive it)
buffer 8 pts 0.14 duration 0.02 at real time t+0.14 (buffer has data from the
sink_0 pad, because it waited enougth time ~0.02 ms to receive it)

Could you please clarify which of two behaviors is correct?

> In live mode, we pop data from the internal queues and mix them. If a source
> is having a huge latency (like in your test, 5s) that is not declared, then
> that source will be ignored from the mixing. 

In my test you can see that the first messages which are late are not ignored
by the mixer, as well as few next messages (~5).
That's what I tried to explain in item 2

> ...but does not seem valid. With RTP, if you have this case, you can increase
> the latency on rtpjitterbuffer element. What is not out-of-the box is this
> latency, we don't have a dynamic latency mode in GStreamer.

I expected that rtpjitterbuffer's latency shows what is a maximum time
rtpjitterbuffer can keep incoming message before it has to be produced. It is
needed for fix some possible issues with message reordering and bursting which
can happend during sending messages over UDP.

Im not sure I can calculate and set latency for RTP and audiomixer, because the
moment when pipeline is ready to receive messages is not the moment when it
receives the first incoming RTP message.
Also it is looks bit strange for me that it is not enougth just prepare
pipeline and wait while it start to receive data.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.