[gst-devel] Re: Distributed pipelines and modular synthesis ramble

Conrad Parker conrad at vergenet.net
Tue Feb 27 20:53:51 CET 2001

On Tue, Feb 27, 2001 at 05:48:52PM +0100, Wim Taymans wrote:
> Here's a possible solution:
> P1 and P2 send attach a timestamp to the media data they produce. The
> timestamps travel along with the media data through the pipelines and
> eventually go over the wire to P3. The timestamps can be changed by some
> intermediate plugins too (a delay, for example). P3 will mix the media
> samples based on the timestamps.
> some things to consider:
>  - global sync of the clock.
>  - delay introduced by the net connection. This will typically create a QoS
>   message that travels from P3 (or intermediate elements) down the wire
>   to P1/P2 to inform them to reduce/augment the number of packets they
>   send out.
>  - state changes. how do we handle them on remote pipelines? the scheduler
>   changes could make it possible to propagate the state change along with the
>   data?
>  - intermachine connections could be handled with a custom element or with
>   something standard like RTP, SCTP?
> In short I'm not afraid of this :) It'll work...

RTP/RTCP provide the following:

	- RTP packets have sequence numbers and a 32-bit timestamp (payload
dependent, ie. GStreamer could define it how it likes)
	- RTCP [RTP Control Protocol, used for delivery monitoring] has
64-bit NTP timestamps, and is used to report back the network delay and
packet loss. ie. this handles your first two concerns

for the state changes:

first up, this data needs to be transmitted reliably. It can either be
transmitted out-of-band (ie. not over RTP) eg. using some TCP mechanism,
or it can be transmitted many times redundantly over RTP to try to make
sure clients get a correct copy of it. nb. this scenario also includes
clients who may connect to an existing server and need to retrieve some
initial state information, ie. this data may need to be transmitted at
times other than state changes.

in RTP apps there are a few ways of handling this kind of data, depending
on its size and how often it changes.

a couple of examples:

for Vorbis over RTP, the codebooks need to be completely transmitted before
any decoding can be done. [unlike MPEG audio, the codebooks are not preset].
The codebooks are rather large, so a few options exist [1]:

   - Including the first set of codebooks in the SDP description
   - Broadcasting a codebook only stream as a second multicast Vorbis
   - Create some method of requesting the codebooks via RTCP
   - Periodically retransmit the headers inline	
   - Interleave the codebook amongst the data

Where the state data is small and changing often, it can be passed in-band.
eg. for transmitting DV streams, data packs containing critical information
about the video format may be passed within the RTP packet. [2]

Finally, as a caching method and to guard against packet loss, a small
amount of relevant state data is usually passed at the beginning of an
RTP packet, eg. for MPEG video, some selected fields from the MPEG frame
headers are replicated at the beginning of every RTP packet [3].  (They
also occur within the RTP payload at the start of each MPEG frame).

[1] Internet-Draft (work in progress): RTP Payload Format for Vorbis
    Encoded Audio, Jack Moffit
    and ensuing discussions on vorbis-dev

[2] Internet-Draft (work in progress): RTP Payload Format for DV Format
    Video <draft-ietf-avt-dv-video-03.txt> [expired?]

[3] RFC 2250: RTP Payload Format for MPEG1/MPEG2 Video


More information about the gstreamer-devel mailing list