[Bug 698851] playbin: ability to mix or play multiple audio and text streams simultaneously

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=698851
  GStreamer | gst-plugins-base | git

--- Comment #9 from Sebastian Dröge <slomo at circular-chaos.org> 2013-05-17 06:46:36 UTC ---
.(In reply to comment #4)
> Here's a potential problem: Playbin creates its input-selectors on-demand, when
> the first stream of that type is added, then removes the input-selector when
> the last stream goes away. It does that because having an input-selector with
> no inputs locks up the pipeline. So, setting a stream combiner as a property on
> playbin before it starts is dangerous.

I don't see the problem :)

The properties would hold an element instance, that is removed/added to the bin
as need (just as is done now with input-selector, just that it's not destroyed
and recreated like input-selector). It shouldn't be a factory name because that
would limit its usefulness (you need to register your private stream combiner
element).

(In reply to comment #6)
> Can't we simply replace the input-selector with a stream-combiner. Then if you
> set current-audio, you end up with a single linked to the combiner and with a
> new property you would to enable multiple audio streams (or none), which would
> make the current-audio to be be -1 ? Would that keep things simple and usable ?
> Because a stream-combiner is a superset of a input-selector.

stream-combiner of encodebin? That's broken in different ways, and Olivier
already proposed to just drop it and replace it by funnel. However this is not
about stream-combiner or input-selector, but allowing custom combination
elements, e.g. adder or videomixer or your fancy bin that does stuff :)


(In reply to comment #7)

> For what I want to do (implement the HTML5 spec), we'll actually need a better
> adder, which synchronized streams and allows them to be turned on and off
> (probably the same element you're thinking of), plus an appsink with dynamic
> pads (presumably easy to build out of normal appsinks), which doesn't need an
> output at all.

That's unrelated from this bug and should definitely be done. Current adder is
only useful in very rare scenarios because it just does not properly
synchronize streams to their running times.

The addition of properties to mute streams or change their volume can easily be
added to adder, and should be added there. Just make them properties on the
sinkpads of adder, similar to what is done in videomixer to specify e.g. the
position of a stream.

> The reason for the *-stream-combiner property is that we don't necessarily want
> to do the same thing with all of the streams.

Yes, there should be one property per stream type.

(In reply to comment #8)
> Oh, and the problem with just replacing input-selector with a generic stream
> combiner is that:
> 
>   * It's not really clear what it would do with videos.

videomixer, you could do some picture-in-picture thing for example.

>   * In my use case (HTML5 spec), I need to know which stream each text cue is
> associated with, so just combining them doesn't work.

One way of doing that would be to have a custom stream combiner for the text,
that muxes all text streams together (so you can split them apart later again)
and then have a custom text-sink that does whatever you want to do with the
different text streams.

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.