[gst-devel] why MPEG fails

[Martin Soto]

Hey Ronald:

On Fri, 2004-03-12 at 17:30, Ronald Bultje wrote:
> audio[0.0s] : video[1.0s] : audio[0.1s] : video[1.1s]
> 
> Those who know scheduling knows what comes next: mpegdemux will spit out
> an audio buffer, mad decodes, sends to osssink, it plays. osssink provides
> the master clock, so the clock is updated from 0.0s to 0.1s. Then, it
> spits out the video buffer, ximagesink does a gst_element_wait() for 1s,
> and displays. During this 1s, nothing happens, because we're
> single-threaded! We go back to audio, video (again 1s wait, because the
> clock-diff is still 1s between video and audio). The result is a 1fps
> movie perception for the user.

I'm not sure I understand your buffer notation, but I guess what happens
is that you reach a deadlock: the audio clock doesn't advance because
there's no more audio, and the video doesn't play because the clock is
actually stopped. You would break the deadlock by sending some more
audio, but that's not possible because the pipeline is already stuck. If
it progresses at all, is only because gst_element_wait gives up at some
point.

Another type of clock wouldn't help either. You can try using the system
clock just for fun, but I bet I can predict the result: very rough
playback with lots of idle waiting and material dropping.

> The MPEG specifications suggest to queue data to prevent this. One way of
> doing that is by using the queue element. This plays fine:

Since the standard itself recommends that, this is what you should do.
And it'd be better to have the buffering explicitly in the pipeline, and
not as a side effect of the scheduler you chose. Our problem right now,
however, is that we cannot queue material without introducing threads.
We need an scheduler that supports that, but before we reach that point,
we're stuck with the threads.

M. S.
-- 
Martin Soto <soto at informatik.uni-kl.de>