What to do when one half of a stream plugs up a preroll?

[Eric Montellese]

Folks, I have the following issue:

I am playing an h264 clip, but not from the very beginning (imagine this
simulates the case of streaming AV input).

What happens is that the first I-frame is lost, which causes the video
decoder to discard the frames that follow.

Meanwhile, the audio path fills up until all of the queues and buffers on
that path are full.  At this point the demux blocks (it can't send any more
buffers), and the audio_sink is blocked waiting for prerolling to complete
so that it can go to the PLAYING state.  Deadlock.

What should I do about this?

I have a few ideas, but none of them are great.
1. We could try to adjust the gstreamer "prerolling" strategy.  (Allow
audio to play even though video is not ready, or similar.  special case
it).  The problem is that doing this is somewhat hackish, and breaks the
general gstreamer "way"
2. We could set up the demux to automatically discard data on one path if
the other path has not prerolled.  This also breaks the gstreamer "way"
though because it requires the demux and sink elements to be aware of each
other.
3. Let the application handle it (detect the stuck state and flush the
audio sink).  But this means that a pipeline constructed with gst-launch
would fail, which is less than ideal.  It's also not the cleanest solution
anyway.
4. We could adjust things to read at the beginning of the input file.  This
is not a real fix though because in the case of streaming data, we would
never know when the connection would be made.  Fixing the lost-data issue
will fix the specific file-based cases, but will not fix the general
problem.
5. Could have some sort of detection element that looks at the whole stream
and chucks data until an I-frame is detected (or the demux could do this
directly).  This would be a big hassle of course.

None of these are great options.  Anything I missed?

Thanks,
Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130305/c6517d7c/attachment.html>