Interleaving two RTP streams

Alexander Kordecki alex at kordecki.de
Tue Aug 7 15:57:44 UTC 2018 

I try to interleave two RTP streams into one file.
The streams can start asynchronously but the content is synchronous.

my current pipeline for testing looks like this:

gst-launch-1.0 -v -t \
    interleave name=i ! \
        audioconvert ! \
        audioresample ! \
        osxaudiosink \
    audiomixer name=mix_left ! audioconvert ! "audio/x-raw, channels=1, channel-mask=(bitmask)0x1" ! i.sink_0 \
    audiomixer name=mix_right ! audioconvert ! "audio/x-raw, channels=1, channel-mask=(bitmask)0x2" ! i.sink_1 \
    audiotestsrc wave=6 name=silence_left is-live=true ! audioconvert ! "audio/x-raw, channels=1, rate=8000, format=S16LE" ! mix_left. \
    audiotestsrc wave=6 name=silence_right is-live=true ! audioconvert ! "audio/x-raw, channels=1, rate=8000, format=S16LE" ! mix_right. \
    udpsrc port=12345 timeout=1 caps="application/x-rtp, media=audio, clock-rate=8000, encoding-name=PCMA, channels=1" name=udp_left ! \
        rtpjitterbuffer do-lost=true ! \
        rtppcmadepay ! \
        alawdec ! \
        audioconvert ! "audio/x-raw, channels=1, rate=8000, format=S16LE" ! \
        mix_left. \
    udpsrc port=12346 timeout=1 caps="application/x-rtp, media=audio, clock-rate=8000, encoding-name=PCMA, channels=1" name=udp_right ! \
        rtpjitterbuffer do-lost=true ! \
        rtppcmadepay ! \
        alawdec ! \
        audioconvert ! "audio/x-raw, channels=1, rate=8000, format=S16LE" ! \
        mix_right.


Starting this pipeline I have noise on both channels just to hear that mixing & interleaving is working.

As soon as I start to send a test RTP stream with:

gst-launch-1.0 audiotestsrc freq=400 is-live=true ! audioconvert ! "audio/x-raw, channels=(int)1, rate=8000" ! alawenc ! rtppcmapay min-ptime=20000000 max-ptime=20000000 ! udpsink host=127.0.0.1 port=12345

my receiver pipline shows an error and the stream is not mixed into the noise:

Leitung wird auf PAUSIERT gesetzt ...
0:00:00.118922000 94630 0x7fd7aa841230 WARN              aggregator gstaggregator.c:1399:gst_aggregator_query_latency_unlocked:<mix_left> Latency query failed
0:00:00.118940000 94630 0x7fd7ac0dfb20 WARN              aggregator gstaggregator.c:1399:gst_aggregator_query_latency_unlocked:<mix_right> Latency query failed
0:00:00.119081000 94630 0x7fd7ac0dfad0 FIXME                default gstutils.c:3825:gchar *gst_pad_create_stream_id_internal(GstPad *, GstElement *, const gchar *):<silence_left:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:00.119114000 94630 0x7fd7aa841280 FIXME                default gstutils.c:3825:gchar *gst_pad_create_stream_id_internal(GstPad *, GstElement *, const gchar *):<silence_right:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Leitung ist aktiv und erfordert keinen VORLAUF …
/GstPipeline:pipeline0/GstOsxAudioSink:osxaudiosink0: device = 196
/GstPipeline:pipeline0/GstUDPSrc:udp_left.GstPad:src: caps = application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)PCMA, channels=(int)1, payload=(int)8
/GstPipeline:pipeline0/GstUDPSrc:udp_right.GstPad:src: caps = application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)PCMA, channels=(int)1, payload=(int)8
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)PCMA, channels=(int)1, payload=(int)8
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer1.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)PCMA, channels=(int)1, payload=(int)8
/GstPipeline:pipeline0/GstAudioTestSrc:silence_left.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioTestSrc:silence_right.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioConvert:audioconvert3.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioConvert:audioconvert4.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter3.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioMixer:mix_left.GstAudioMixerPad:sink_0: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioConvert:audioconvert3.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioMixer:mix_right.GstAudioMixerPad:sink_0: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter3.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
/GstPipeline:pipeline0/GstAudioConvert:audioconvert4.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1
Leitung wird auf ABSPIELEN gesetzt ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer1.GstPad:src: caps = application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)PCMA, channels=(int)1, payload=(int)8
/GstPipeline:pipeline0/GstRtpPcmaDepay:rtppcmadepay0.GstPad:src: caps = audio/x-alaw, channels=(int)1, rate=(int)8000
/GstPipeline:pipeline0/GstRtpPcmaDepay:rtppcmadepay1.GstPad:src: caps = audio/x-alaw, channels=(int)1, rate=(int)8000
/GstPipeline:pipeline0/GstALawDec:alawdec0.GstPad:sink: caps = audio/x-alaw, channels=(int)1, rate=(int)8000
/GstPipeline:pipeline0/GstALawDec:alawdec0.GstPad:sink: caps = audio/x-alaw, channels=(int)1, rate=(int)8000
/GstPipeline:pipeline0/GstALawDec:alawdec0.GstPad:sink: caps = audio/x-alaw, channels=(int)1, rate=(int)8000
/GstPipeline:pipeline0/GstRtpPcmaDepay:rtppcmadepay0.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)PCMA, channels=(int)1, payload=(int)8
0:00:00.411604000 94630 0x7fd7ac0dfb20 FIXME               basesink gstbasesink.c:3125:gboolean gst_base_sink_default_event(GstBaseSink *, GstEvent *):<osxaudiosink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:00.420893000 94630 0x7fd7ac0dfb20 WARN         audio-resampler audio-resampler.c:362:void convert_taps_gint16_c(gdouble *, gpointer, gdouble, gint): can't find exact taps
0:00:00.425565000 94630 0x7fd7aa841230 FIXME             interleave interleave.c:917:gst_interleave_sink_event:<i> FIXME: merge tags and send after stream-start
0:00:00.425776000 94630 0x7fd7ac0dfb20 FIXME             interleave interleave.c:917:gst_interleave_sink_event:<i> FIXME: merge tags and send after stream-start
/GstPipeline:pipeline0/GstAudioMixer:mix_left.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000001
/GstPipeline:pipeline0/GstAudioMixer:mix_right.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000002
/GstPipeline:pipeline0/GstAudioConvert:audioconvert1.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000001
/GstPipeline:pipeline0/GstAudioConvert:audioconvert2.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000002
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000001
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000002
/GstPipeline:pipeline0/GstInterleave:i.GstInterleavePad:sink_0: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000001
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000001
/GstPipeline:pipeline0/GstInterleave:i.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioResample:audioresample0.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
Verzögerung neu verteilen …
/GstPipeline:pipeline0/GstOsxAudioSink:osxaudiosink0.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioResample:audioresample0.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
KENNZEICHEN GEFUNDEN: Gefunden von Element »osxaudiosink0«.
    Beschreibung: audiotest wave
/GstPipeline:pipeline0/GstInterleave:i.GstInterleavePad:sink_1: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000002
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000002
/GstPipeline:pipeline0/GstAudioConvert:audioconvert2.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)8000, format=(string)S16LE, channels=(int)1, channel-mask=(bitmask)0x0000000000000002
KENNZEICHEN GEFUNDEN: Gefunden von Element »osxaudiosink0«.
    Beschreibung: audiotest wave

####
#### —> in this moment I started the stream
####


0:00:04.280115000 94630 0x7fd7aa841190 WARN                GST_PADS gstpad.c:4100:gboolean gst_pad_peer_query(GstPad *, GstQuery *):<capsfilter4:src> could not send sticky events
0:00:04.281680000 94630 0x7fd7aa841190 WARN                GST_PADS gstpad.c:4100:gboolean gst_pad_peer_query(GstPad *, GstQuery *):<capsfilter4:src> could not send sticky events
0:00:04.316226000 94630 0x7fd7aa841320 WARN                 basesrc gstbasesrc.c:2947:void gst_base_src_loop(GstPad *):<udp_left> error: Interner Fehler im Datenfluss.
0:00:04.316310000 94630 0x7fd7aa841320 WARN                 basesrc gstbasesrc.c:2947:void gst_base_src_loop(GstPad *):<udp_left> error: streaming task paused, reason not-negotiated (-4)

####

^Chandling interrupt.
/GstPipeline:pipeline0/GstALawDec:alawdec0.GstPad:src: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)8000, channels=(int)1
/GstPipeline:pipeline0/GstAudioConvert:audioconvert5.GstPad:src: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)8000, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter4.GstPad:src: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)8000, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter4.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)8000, channels=(int)1
/GstPipeline:pipeline0/GstAudioConvert:audioconvert5.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)8000, channels=(int)1
FEHLER: Von Element /GstPipeline:pipeline0/GstUDPSrc:udp_left: Interner Fehler im Datenfluss.
Zusätzliche Fehlerdiagnoseinformation:
gstbasesrc.c(2947): void gst_base_src_loop(GstPad *) (): /GstPipeline:pipeline0/GstUDPSrc:udp_left:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:29.256633000
Leitung wird auf PAUSIERT gesetzt ...
Leitung wird auf BEREIT gesetzt ...
Leitung wird auf NULL gesetzt ...
Leitung wird geleert ...



any ideas how to solve this ?

Regards,
alex

More information about the gstreamer-devel mailing list