[gst-devel] rtp - mux or not to mux?
Kai Vehmanen
kvehmanen at eca.cx
Wed Nov 2 04:22:33 CET 2005
Hello all,
continuing the rtp-invasion of gstreamer-devel. :) Earlier I posted an
high-level proposal about my gstsdpbin idea:
http://article.gmane.org/gmane.comp.video.gstreamer.devel/13940
Now I'm working on the details of the demuxing/switching part, in other
words, how to support multiple codecs inside one stream of RTP packets
(RTP session). This should be useful for various uses of RTP in
gstreamer apps...
The problem is somewhat similar to mpeg/ogg demuxing, but with few key
differences:
- an RTP stream will only contain one logical stream;
the codec might change, but the packets will have different
timestamps and sequence numbers (different codecs do
not overlap timestamp wise, as happens with ogg/mpeg and
other container streamer)
=> the reception pipeline is really 1:1
- in most cases, you will know the possible subcomponents
beforehand; and actually, with many codecs it's actually
impossible to identify the codec type by looking at the RTP packets
=> the application has to specify which codecs, and
pt<->codec mappings to use in parsing the incoming stream
I've been studying the current demuxers (and other matching elements in
gstreamer), and it seems that this could be implemented in gstreamer
either as:
1) rtpdemux -> 1:N demux element (like oggdemux for example)
- connected to rtpbin/rtprecv's src pad
- as auto-detection is not possible, client will have to
setup pads beforehand (use PT=x for G711, PT=x for speex,
and so on; static PTs are an exception here)
- the mux will look at incoming packets, and push the
them forward to the matching src pad
- the pipeline:
udpsrc -> rtprecv -> rtpdemux -> depayloader(s) -> decoder(s) -> sink
- possible problems:
- in the audio case, I guess "adder" wouldn't be enough to mux the
multiple decoder paths together as it requires input from
all sources before mixing
- a modified version of "adder"? .. and same for video
2) rtp-switcher -> 1:1 element that can decode and depacketize multiple codecs
- as (1), connected to rtpbin/rtprecv's src pad
- creates and manages multiple depayloader+decoder elements
- each received packet is pushed through exactly one
depayloader+decoder combo
- sink pad takes application/x-rtp, and pushes out
encoded media (e.g. audio/x-raw-int) on its src pad
- possible problems:
- how to implement in gstreamer? any good example elements
that are doing something like this?
Now any comments on which approach is best from gstreamer architecture
point of view?
One problem common to both approaches is the jitter buffer code, now
in gst-plugins-base-cvs/gst-libs/gst/rtp/gstbasertpdepayload.c. Ideally
different codecs should share the same jitter buffer (so that change of
codec mid-stream wouldn't result in throwing out the whole jitter buffer),
but this is pretty hard to realize with the current elements... or?
--
More information about the gstreamer-devel
mailing list