[gst-devel] why MPEG fails

[Ronald S. Bultje]

Hi Martin,

On Fri, 2004-03-12 at 22:22, Martin Soto wrote:
> On Fri, 2004-03-12 at 17:30, Ronald Bultje wrote:
> > audio[0.0s] : video[1.0s] : audio[0.1s] : video[1.1s]
> > 
> > Those who know scheduling knows what comes next: mpegdemux will spit out
> > an audio buffer, mad decodes, sends to osssink, it plays. osssink provides
> > the master clock, so the clock is updated from 0.0s to 0.1s. Then, it
> > spits out the video buffer, ximagesink does a gst_element_wait() for 1s,
> > and displays. During this 1s, nothing happens, because we're
> > single-threaded! We go back to audio, video (again 1s wait, because the
> > clock-diff is still 1s between video and audio). The result is a 1fps
> > movie perception for the user.
> 
> I'm not sure I understand your buffer notation, but I guess what happens
> is that you reach a deadlock: the audio clock doesn't advance because
> there's no more audio, and the video doesn't play because the clock is
> actually stopped. You would break the deadlock by sending some more
> audio, but that's not possible because the pipeline is already stuck. If
> it progresses at all, is only because gst_element_wait gives up at some
> point.

No, no. This is the actual buffer flow. No thread, so all's lineair.
Just go through the file like an MPEG demuxer.

* read audio buffer
* decode
* put in osssink
* osssink sets audio clock to time on buf (0s)
* read video buffer
* decode
* put in ximagesink
* gst_element_wait(1s)
* triggers a wait for 1s (clock diff between clock and buffer)
* play frame
* read audio buffer
[.. etc.]

So we end up playing the movie at 1fps.

> > The MPEG specifications suggest to queue data to prevent this. One way of
> > doing that is by using the queue element. This plays fine:
> 
> Since the standard itself recommends that, this is what you should do.
> And it'd be better to have the buffering explicitly in the pipeline, and
> not as a side effect of the scheduler you chose. Our problem right now,
> however, is that we cannot queue material without introducing threads.
> We need an scheduler that supports that, but before we reach that point,
> we're stuck with the threads.

I'd love to do this queueing internally (since it's a problem unique to
MPEG), but that'll take some more effort. That won't be done before
0.8.0. Until then, I don't care using a hack to fix this.

Ronald