[gst-devel] syncronizing multiple pipelines

[Stefan Kost]

Attila RS wrote:
> I'm working on a app that needs to dynamically add/remove outputs
> without affecting already active outputs. Below is a simple example.
>
> 1. Receive an RTP stream with audio/video and playback both.
> 2. Stop audio output associated with the above stream.
> 3. Receive a 2nd RTP stream with audio and playback audio.
> 4. Stop 2nd RTP stream.
> 5. Connect audio output back to the first RTP stream without affecting
> the video.
>
> In order to be able to dynamically add/remove outputs I
> have separated the input (udpsrc/depay/demux), the video output
> (decode/vidsink) and the audio output (decode/audsink) into
> 3 separate pipelines and link them together with ghost pads. This
> seems to work well for doing the above controls, but I have 2 problems. 
>
> 1. Synchronization is a mess. It looks like each pipeline is using a
> different clock and as a result audio/video is out of sync. 
> 2. Adding a new output pipeline to a running input pipeline result in
> out of sync clocks, so the newly added output thinks it is too far
> ahead/behind and just dumps all data instead of outputting it.
> Disabling sync and the newly added pipeline starts playing fine, but
> of course it is out of sync.
>
> I think the second problem can probably be improved or eliminated by
> forcing a new segment event when linking the output, but still the
> clocks need to be synchronized between the pipelines in order for a/v
> to sync correctly and playback smoothly.
>
> Any thoughts on how to better handle these problems? Maybe forcing the
> output pipelines to use the input pipelines clock? Or perhaps there is
> a way to maintain the control I need without using 3 different pipelines?

You want to have one pipeline, so that you share *one* clock. You can
dynamicly link and unlink elements. Read part-blocking.txt in the source
tree under gstreamer/docs/design. You need to use pad-blocking to ensure
no dataflow is happening on the unlinked pads. If a sink is not
conencted its probably a good idea to pause it (and lock the
paused_state). If you don't use the audiosink, you need to trigger clock
re-selection (dunno how from the top of my head - maybe you can check
what playbin2 does if you unset the audio flag). When connecting a new
upstream branch to a sink, you need to resend the newsegment event to
the sink so that it 'knows' whats playing.

I know that all this is a bit complicated. Making this easier on the
framework level would be nice ...

Stefan


>
> Thanks for the help.
>
> Attila
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ------------------------------------------------------------------------
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>