What to do when one half of a stream plugs up a preroll?

[Eric Montellese]

It seems like this would be the sort of issue that others have run
into -- anyone have any advice, or experience?


On Tue, Mar 5, 2013 at 5:58 PM, Eric Montellese
<eric.montellese at videon-central.com> wrote:
> Folks, I have the following issue:
>
> I am playing an h264 clip, but not from the very beginning (imagine this
> simulates the case of streaming AV input).
>
> What happens is that the first I-frame is lost, which causes the video
> decoder to discard the frames that follow.
>
> Meanwhile, the audio path fills up until all of the queues and buffers on
> that path are full.  At this point the demux blocks (it can't send any more
> buffers), and the audio_sink is blocked waiting for prerolling to complete
> so that it can go to the PLAYING state.  Deadlock.
>
> What should I do about this?
>
> I have a few ideas, but none of them are great.
> 1. We could try to adjust the gstreamer "prerolling" strategy.  (Allow audio
> to play even though video is not ready, or similar.  special case it).  The
> problem is that doing this is somewhat hackish, and breaks the general
> gstreamer "way"
> 2. We could set up the demux to automatically discard data on one path if
> the other path has not prerolled.  This also breaks the gstreamer "way"
> though because it requires the demux and sink elements to be aware of each
> other.
> 3. Let the application handle it (detect the stuck state and flush the audio
> sink).  But this means that a pipeline constructed with gst-launch would
> fail, which is less than ideal.  It's also not the cleanest solution anyway.
> 4. We could adjust things to read at the beginning of the input file.  This
> is not a real fix though because in the case of streaming data, we would
> never know when the connection would be made.  Fixing the lost-data issue
> will fix the specific file-based cases, but will not fix the general
> problem.
> 5. Could have some sort of detection element that looks at the whole stream
> and chucks data until an I-frame is detected (or the demux could do this
> directly).  This would be a big hassle of course.
>
> None of these are great options.  Anything I missed?
>
> Thanks,
> Eric