[gst-devel] Proposition for a media test suite

[Wim Taymans]

On Mon, 2005-11-14 at 13:21 -0500, Ronald S. Bultje wrote: 
> On Mon, 2005-11-14 at 18:50 +0100, Wim Taymans wrote:
> > Filler events are a different beast, they are mainly needed when dealing
> > with sparse streams to insert a long period of no buffer activity that
> > might stall a pipeline. I hope with the new scheduling in 0.9 we don't
> > need them but they're still there just in case they're needed after all.
> > No plugins need this yet, so...
> 
> Subtitles will need it, even with the current scheduling model. DVD
> menus will, most likely, need it also, although I haven't looked into
> that for 0.9 yet (and will not anytime soon).

Use case 1:

subtitles from separate file (avi, some others..)

             ...      -> video decoder ->
                                           video-mixer -> videosink
filesrc -> subtitle parser -> renderer ->

since the video decoder and filesrc run in a separated thread, the
subtitles can be queued in video-mixer and displayed when the video
frames pass by, when the duration of the subtitle expires, the
video-mixer waits for a new subtitle to merge. Since video-mixer is
chain-based (possibly with collectpads) and video/subtitles are
completely decoupled, nothing locks up or times out, no filler events
needed. We needed to have them in 0.8 because both sources could not be
scheduled decoupled. If the video-mixer is pure chain-based, we could be
racing against the kernel scheduler and render a subtitle too late.

Use case 2:

subtitles in DVD.

            -> video decoder     ->
-> demuxer                         video-mixer -> videosink
            -> subtitle renderer ->

The demuxer pushes out video and subtitles in the same thread (although
a queue should ideally be placed in front of the video-decoder). Here
the video-mixer can operate in chain-based mode. Whenever it receives a
subtitle, it stores it and renders it when the corresponding
video-frames pass by. When the duration of the subtitle expires or a new
subtitle is chained, it renders the new subtitle to the video-frames.
This assumes video-mixer is not using collectpads. If video-mixer was
using collectpads, a filler event would be needed to make sure video is
mixed even if there is no subtitle yet. 

So, case 1 ideally needs a collectpads video-mixer, case 2, ideally a
pure chain-based mixer. Not sure yet how to signal the scheduling mode
to the mixer or maybe we need two kinds of mixers.. 

Other uses of fillers in 0.8 where to make the audioclock advance when
no audio was fed to the audiosink, which is not needed anymore in 0.9.
This used to be the case for DVD menus, for example.

Wim

> 
> Ronald
> 
-- 
Wim Taymans <wim at fluendo.com>