[Bug 736655] basesink: preroll issue for some clips which audio is shorter than video

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=736655
  GStreamer | gstreamer (core) | 1.2.3

--- Comment #96 from Jan Schmidt <thaytan at noraisin.net> 2015-02-05 15:13:17 UTC ---
(In reply to comment #95)
> (In reply to comment #94)
> > (In reply to comment #93)
> > > (In reply to comment #92)
> > > > My latest patches for audiobasesink in Bug 735666 help with the prerolling
> > > > here, but only up to a point - audiobasesink will process a GAP with timestamp
> > > > & duration as if it were an audio buffer of the same time+duration and
> > > > successfully go PLAYING->PAUSED->PLAYING.
> > > > 
> > > How about video sink? Do video sink need GAP event to finish pre-roll when
> > > PLAYING->PAUSED?
> > 
> > The same thing applies to video sinks - they need a GAP to preroll too.
> > 
> If video is shorter than audio, streamsynchronizer will send GAP event when
> received video EOS. How to set the GAP duration?

The duration can be 0 or GST_CLOCK_TIME_NONE, it's fine - the key is that a pad
that reaches EOS early should loop in EOS sending gap events downstream with
advancing timestamps taken from the other non-EOS stream(s), to keep downstream
updated on progress until everything reaches EOS.

The gap events it sends should be based on the maximum position of the other
stream(s) - buffer times + duration. The existing logic that calculates a start
time for the gap event it sends now is probably fine.

> If pipeline change state from PLAYING to PAUSED, do streamsynchronizer need
> send GAP event again to finish video pre-roll? do streamsynchronizer need send
> GAP event every time when pipeline change state from PLAYING to PAUSED?
> > > > That means that streamsynchroniser needs to make sure to send a GAP (or GAPs)
> > > > that fill the entire time range until it is ready to either start a new stream
> > > > or send EOS.
> > > > 
> > > streamsynchronizer will seng GAP with duration is GST_CLOCK_TIME_NONE. Do
> > > streamsychronizer need search all segment to get to biggest time range? Or
> > > audio sink can handle GST_CLOCK_TIME_NONE.
> > 
> > Duration GST_CLOCK_TIME_NONE is handled as duration = 0 in the sink, so it will
> > effectively 'fill time' up to the start timestamp.
> > 
> So streamsynchronizer should send GAP with duration is GST_CLOCK_TIME_NONE when
> received EOS for shorter track? and send GAP event for the shorter track to
> 'moves time forward' in other track chain function? And send all tracks EOS
> event after received all tracks EOS event.

Yes, if I've understood what you're trying to say correctly.

> > > > It looks like these patches will achieve that, but sending a new GAP when
> > > > changing state in stream synchroniser, but wouldn't it be better to wake the
> > > > streams that are looping on EOS every time data passes on another pad that
> > > > 'moves time forward' and have them send another GAP event then?
> > > > 
> > > streamsynchronizer will send GAP event in chain function. But pad push will be
> > > blocked when sink changed to PAUSED state and then can't send GAP event. So
> > > need send GAP event when state change in streamsynchronizer.
> > 
> > If pad_push() is blocked, you can't send a GAP event downstream there anyway -
> > the STREAM_LOCK() will be held by the pad_push(). That case should work. The
> > one that is broken is where you sent a GAP downstream from teh EOS event
> > handler, advancing time to some time 'timestamp', and then time advanced past
> > that point, and your pad_push_event (pad, gap_event) returned - you need to
> > either push a new gap, or push a buffer then.
> 
> If pad_push() is blocked, streamsynchronizer can't send GAP event for shorter
> track in chain function. The shorter track is reach EOS, and no any buffer
> reach streamsynchronizer, so basesink can't finished pre-roll for the shorter
> track. The longer track is pro-rolled and blocked. So streamsynchronizer need
> send GAP event for shorter track when state change, and send the GAP event in
> event handle thread.

There are 2 threads / streams in stream synchroniser:

* Short stream, has reached EOS. It pushes a gap downstream using
gst_pad_push_event(), which won't return until the end time (timestamp +
duration) of the gap event. Once that returns, it is free to push another gap
event - or stop pushing them if it's time to switch to another input.
* The longer stream, buffers are still passing, and can be used to set
timestamps on the gap events being sent in the other thread.

Is that clearer?

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.