[Bug 744106] Maximum latency handling broken everywhere

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=744106
  GStreamer | gstreamer (core) | unspecified

Nicolas Dufresne (stormer) <nicolas.dufresne> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |nicolas.dufresne at collabora.
                   |                            |co.uk

--- Comment #2 from Nicolas Dufresne (stormer) <nicolas.dufresne at collabora.co.uk> 2015-02-07 16:56:25 UTC ---
I just re-read the design doc. I think we need to see why this is an impossible
case:

  sink1: [20 - 20]
  sink2: [33 - 40]

  MAX (20, 33) = 33
  MIN (20, 40) = 20 < 33 -> impossible

sink1 query and sink2 query could be in a graph or two completly seperate
pipeline in the same bin, it would not matter. Basically, what this query says,
is that from sink1 point of view, the pipeline won't be able to hold on more
then 20ms of multimedia data. While on sink2, it can hold up to 40ms of data.

What the min latency says is that to be synchronise these stream, both sink
need to reach a target latency of 33ms. So sink1 has no mean to synchronise
both stream, since it can hold on enough data to compensate the other sink
delay.

At least that's what the doc seems to be trying to say. In this regard, max
latency has nothing to do with blocking, it's in fact the amount of data that
can be buffered over all the pipeline. That includes the latency of course and
that's a good explanation of why in element contribution to latency, mininum
contribution should never be bigger then maximum.

If we take x264enc with it's latest patch, we would have:

livesrc [10 - 10] ! x264 [5 - inifnit] ! tee name=t
  t. ! sink1 [5 - 5]
  t. ! sink2 [18 - 25]

Previously, x264 would have set [5 - 5], which would have given a resulting
[33, 20], an impossible situation. Now with the path, we get [33, infinit] a
properly working pipeline.

Now sadly I'd don't known the internal behaviour of x264, but if that is true,
it means x264 has an infinit input queue. Great, since it solve the issue.

But, One important fact about x264 element though is that it uses
GstVideoDecoder base class. This class has this interesting limitation, is that
it blocks input while pushing outputs. Considering we are live, input and
output flow have very similar speed, hence it is likely that after the minimum
latency has been reach, each input buffer will produce an output buffer (I
voluntarily wave out interlacing here).

What this mean, is that whatever internal implementation details of x264, the
element will never accumulate an inifit amount of buffer. It will in fact
block. The result is that the sink1 will have accumulate 5ms, the livesrc 10
and x264 5ms (maybe plus 1 if we are lucky). This is 20ms, but we need to hold
on 33ms, it's in fact a situation impossible to synchronise.

In this regard, what would have failed here, will fail differently with the
current change. So in this case, deprecating max latency, that would server at
sending an eror "please add a queue in your pipeline", would in fact be
expressed as a pipeline with a massive amount of data being dropped, because
there is no where to store them while delaying the playback.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.