Using input-selector

[Jan Alexander Steffens]

On Sun, Jan 19, 2014 at 10:56 AM, Sebastian Dröge
<sebastian at centricular.com> wrote:
> On Do, 2014-01-16 at 09:55 +0100, Jan Alexander Steffens wrote:
>> Hello List,
>>
>> I'm trying to use input-selector to switch between two streams.
>> Streams that are unconnected should be paused (in case of a file) or
>> continue running as normal (except without output; in case of a
>> livestream). How do I do this? I think my previous attempts fail
>> because the sink gets confused when the timestamps suddenly change.
>>
>> For the big picture; I'm trying to get the bits and pieces together to
>> build a live video composer.
>
> That's actually not so trivial to do unfortunately. The default case is
> that input-selector just drops data from every unselected stream, so
> that should be no problem.
>
> However pausing the file-based streams is, as the pausing is usually
> only implemented in the sinks and for other elements there's usually no
> difference between PLAYING and PAUSED state. Also you will need to
> adjust the timestamp after the pausing to be at the exact current
> running time, not the running time when you paused. For the latter you
> should be able to use the pad offsets (gst_pad_set_offset()), for the
> pausing you could just block the pads of the file-based streams
> (gst_pad_add_probe() with a blocking probe type).

I've restricted myself to live sources at the moment and I've managed
to produce a prototype that switches back and forth between two video
streams coming from RTMP sources.

Now I want to do the same with audio, too. My naive approach simply used
two input-selectors, however the switch is not simultaneous—even though
I'm switching the audio selector first, the audio switch is late by
half a second
or so, probably because the flow through the selectors is not synchronized.

Will I need to implement my own multi-input-selector (-bin?) to synchronize the
switch properly? I imagine this involves blocking flow through the early
streams until the late ones have caught up.

I've also tried putting a multiqueue in front of the selectors -
however I'm pretty
sure I'm not using it right as I'm back to my old problem of the output hanging
after the switch.