[gst-devel] Re: Distributed pipelines and modular synthesis ramble

[Wim Taymans]

<snip>

> Now 1) is much easier to implement than 2) because it just needs someone to
> write the bridge source-sink pair and instant distribution. However I see a
> big problem here. Consider this situation (sorry for all the ASCII art!):
>
>s P1===S1s . . S3=
>sssssssssssssssss ==P3
>ss P2===S2s . . S4=/
>
> This time the network is spread accross three diferent machines. My concern
> is that since the S's don't really know anything about the P's there will
> be a serious synchronisation problem. Supposing P1 and P2 are two audio
> sources which are to be mixed by P3. I can't see how S3 and S4 will be able
> to synchronise their inputs so that the P1 and P2 signals are aligned
> properly in the mix. The same would apply to the render farm example.

Here's a possible solution:

P1 and P2 send attach a timestamp to the media data they produce. The
timestamps travel along with the media data through the pipelines and
eventually go over the wire to P3. The timestamps can be changed by some
intermediate plugins too (a delay, for example). P3 will mix the media
samples based on the timestamps.

some things to consider:

 - global sync of the clock.
 - delay introduced by the net connection. This will typically create a QoS
  message that travels from P3 (or intermediate elements) down the wire
  to P1/P2 to inform them to reduce/augment the number of packets they
  send out.
 - state changes. how do we handle them on remote pipelines? the scheduler
  changes could make it possible to propagate the state change along with the
  data?
 - intermachine connections could be handled with a custom element or with
  something standard like RTP, SCTP?

In short I'm not afraid of this :) It'll work...

Wim Taymans