decklinkaudiosrc pitches an error when extracting 8 channels of audio
bishop-00median at icloud.com
bishop-00median at icloud.com
Thu Mar 31 18:44:03 UTC 2022
Thanks. This issue is now resolved. rtpopuspay requires a channel-mask other than 0x0.
Patrick
> On Mar 29, 2022, at 3:57 PM, Patrick Cusack <patrickcusack at mac.com> wrote:
>
> Thanks for your patience,
>
> I can confirm that I did make a mistake by not specifying 'encoding-name=multiopus’ in the caps after rtpopuspay.
>
> That being said, the following ‘ decklinkaudiosrc' still does not work even when I specify 'encoding-name=multiopus':
>
> webrtcbin name=sendonly bundle-policy=3 stun-server=stun://stun.l.google.com:19302 decklinkvideosrc device-number=0 mode=1080p2398 connection=sdi ! videoconvert ! vp9enc target-bitrate=4000000 keyframe-max-dist=24 deadline=1 end-usage=1 cpu-used=8 lag-in-frames=0 ! rtpvp9pay pt=100 ssrc=2 ! application/x-rtp,media=video,encoding-name=VP9,payload=100 ! sendonly. decklinkaudiosrc device-number=0 connection=embedded channels=8 ! queue ! opusenc ! rtpopuspay pt=111 ssrc=1 ! application/x-rtp,media=audio,encoding-name=multiopus,payload=111 ! sendonly.
>
>
> I think that there is something wrong with the caps into the rtpopuspay:
>
> 0:00:00.112774285 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:5757:pre_eventfunc_check:<rtpopuspay0:sink> caps audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > not accepted
>
> Please see the following error:
>
> 0:00:00.017981426 29895 0x558932754a70 WARN decklink gstdecklink.cpp:1525:init_devices: selected device does not have output interface: 0xffffffff80000004
> 0:00:00.018360576 29895 0x558932754a70 WARN decklink gstdecklink.cpp:1525:init_devices: selected device does not have output interface: 0xffffffff80000004
> [WHIP] Starting the GStreamer pipeline
> 0:00:00.018603261 29895 0x5589326efaa0 FIXME default gstutils.c:4025:gst_pad_create_stream_id_internal:<decklinkvideosrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
> [WHIP] Set the GStreamer pipeline to playing
> [WHIP] Returning TRUE
> [WHIP] beginning g_main_loop_run
> 0:00:00.018604971 29895 0x5589326efb00 FIXME default gstutils.c:4025:gst_pad_create_stream_id_internal:<decklinkaudiosrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
> 0:00:00.109785325 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<rtpopuspay0:sink> caps: audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > were not compatible with: EMPTY
> 0:00:00.109821315 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:5757:pre_eventfunc_check:<rtpopuspay0:sink> caps audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > not accepted
> 0:00:00.109859057 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<rtpopuspay0:sink> caps: audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > were not compatible with: EMPTY
> 0:00:00.109877771 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:5757:pre_eventfunc_check:<rtpopuspay0:sink> caps audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > not accepted
> 0:00:00.109882929 29895 0x5589326efa40 WARN GST_PADS gstpad.c:4351:gst_pad_peer_query:<opusenc0:src> could not send sticky events
> 0:00:00.112629567 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<rtpopuspay0:sink> caps: audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > were not compatible with: EMPTY
> 0:00:00.112653893 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:5757:pre_eventfunc_check:<rtpopuspay0:sink> caps audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > not accepted
> 0:00:00.112693847 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<rtpopuspay0:sink> caps: audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > were not compatible with: EMPTY
> 0:00:00.112713128 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:5757:pre_eventfunc_check:<rtpopuspay0:sink> caps audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > not accepted
> 0:00:00.112755303 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<rtpopuspay0:sink> caps: audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > were not compatible with: EMPTY
> 0:00:00.112774285 29895 0x5589326efa40 WARN GST_CAPS gstpad.c:5757:pre_eventfunc_check:<rtpopuspay0:sink> caps audio/x-opus, rate=(int)48000, channels=(int)8, channel-mapping-family=(int)255, stream-count=(int)8, coupled-count=(int)0, channel-mapping=(int)< 0, 1, 2, 3, 4, 5, 6, 7 >, streamheader=(buffer)< 4f707573486561640108380180bb00000000ff08000001020304050607, 4f707573546167731e000000456e636f6465642077697468204753747265616d6572206f707573656e630000000001 > not accepted
> 0:00:00.137734698 29895 0x5589326efaa0 WARN vpxenc gstvpxenc.c:1860:gst_vpx_enc_set_format:<vp9enc0> Failed to set VP8E_SET_NOISE_SENSITIVITY: incapable
> 0:00:00.138080464 29895 0x5589326efaa0 WARN vpxenc gstvpxenc.c:1880:gst_vpx_enc_set_format:<vp9enc0> Failed to set VP8E_SET_TOKEN_PARTIONS: incapable
> 0:00:00.150274770 29895 0x5589326efb00 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<decklinkaudiosrc0> error: Internal data stream error.
> 0:00:00.150291132 29895 0x5589326efb00 WARN basesrc gstbasesrc.c:3127:gst_base_src_loop:<decklinkaudiosrc0> error: streaming stopped, reason not-negotiated (-4)
> 0:00:00.150355145 29895 0x5589326efb00 WARN queue gstqueue.c:992:gst_queue_handle_sink_event:<queue0> error: Internal data stream error.
> 0:00:00.150359295 29895 0x5589326efb00 WARN queue gstqueue.c:992:gst_queue_handle_sink_event:<queue0> error: streaming stopped, reason not-negotiated (-4)
> 0:00:00.359101934 29895 0x7f9130006e60 WARN decklinkvideosrc gstdecklinkvideosrc.cpp:865:gst_decklink_video_src_got_frame:<decklinkvideosrc0> Starting to drop frames
> 0:00:00.400781318 29895 0x7f9130006e60 WARN decklinkaudiosrc gstdecklinkaudiosrc.cpp:557:gst_decklink_audio_src_got_packet:<decklinkaudiosrc0> Starting to drop audio packets
>
>
>> On Mar 29, 2022, at 12:24 PM, Patrick Cusack <patrickcusack at mac.com> wrote:
>>
>> Here is a pipeline that eliminates the ‘decklinkaudiosrc’. The following pipeline fails as well.
>>
>> gst-launch-1.0 -v webrtcbin name=sendonly bundle-policy=3 stun-server=stun://stun.l.google.com:19302 videotestsrc pattern=snow ! videoconvert ! vp9enc target-bitrate=4000000 keyframe-max-dist=24 deadline=1 end-usage=1 cpu-used=8 lag-in-frames=0 ! rtpvp9pay pt=100 ssrc=2 ! queue ! application/x-rtp,media=video,encoding-name=VP9,payload=100 ! sendonly. audiotestsrc wave=7 freq=200 is-live=true ! audio/x-raw,channels=8 ! queue ! opusenc ! rtpopuspay pt=111 ssrc=1 ! queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=111 ! sendonly.
>>
>> If change audiotestsrc caps to channels = 2, then it works. This is the same behavior as the ‘decklinkaudiosrc’. Am I wring to think the error is in the caps?
>>
>> Patrick
>>
>>
>>> On Mar 28, 2022, at 3:39 PM, Patrick Cusack <patrickcusack at mac.com> wrote:
>>>
>>> Thanks. decklinaudiosrc only accepts 2,8 our zero channels. No matter, I looked at the audiomixmatrix and I think that will work.
>>>
>>> As I understand 1.16 doesn’t support multiopus which was added on 1.20. I built 1.20 on Ubuntu 20.04 (as I didn’t see any repos) using Cerbero.
>>>
>>> Here is my is pipeline with the following error:
>>>
>>> gst-launch-1.0 -v webrtcbin name=sendonly bundle-policy=3 \
>>> stun-server=stun://stun.l.google.com:19302 \
>>> decklinkvideosrc device-number=0 mode=1080p2398 connection=sdi \
>>> ! videoconvert \
>>> ! vp9enc target-bitrate=4000000 keyframe-max-dist=24 deadline=1 end-usage=1 cpu-used=8 lag-in-frames=0 \
>>> ! rtpvp9pay pt=100 ssrc=2 \
>>> ! queue \
>>> ! application/x-rtp,media=video,encoding-name=VP9,payload=100 \
>>> ! sendonly. \
>>> decklinkaudiosrc device-number=0 connection=embedded channels=8 \
>>> ! audioconvert \
>>> ! audiomixmatrix in-channels=8 out-channels=6 channel-mask=-1 matrix="<<1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0>, <0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0>, <0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0>, <0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0>, <0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0>, <0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0>>" \
>>> ! audio/x-raw,channels=6 \
>>> ! audioresample \
>>> ! queue \
>>> ! opusenc \
>>> ! rtpopuspay pt=111 ssrc=1 \
>>> ! queue \
>>> ! application/x-rtp,media=audio,encoding-name=OPUS,payload=111 \
>>> ! sendonly.
>>>
>>> Setting pipeline to PAUSED ...
>>> Pipeline is live and does not need PREROLL ...
>>> /GstPipeline:pipeline0/GstDecklinkVideoSrc:decklinkvideosrc0: hw-serial-number =
>>> /GstPipeline:pipeline0/GstDecklinkAudioSrc:decklinkaudiosrc0: hw-serial-number =
>>> Pipeline is PREROLLED ...
>>> Setting pipeline to PLAYING ...
>>> New clock: GstSystemClock
>>> /GstPipeline:pipeline0/GstDecklinkAudioSrc:decklinkaudiosrc0.GstPad:src: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)48000, channels=(int)8, channel-mask=(bitmask)0x0000000000000000
>>> /GstPipeline:pipeline0/GstDecklinkVideoSrc:decklinkvideosrc0: signal = true
>>> /GstPipeline:pipeline0/GstDecklinkVideoSrc:decklinkvideosrc0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, framerate=(fraction)24000/1001, format=(string)UYVY, colorimetry=(string)bt709, chroma-site=(string)mpeg2
>>> /GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)24000/1001, pixel-aspect-ratio=(fraction)1/1, format=(string)Y444, interlace-mode=(string)progressive, colorimetry=(string)bt709
>>> ERROR: from element /GstPipeline:pipeline0/GstDecklinkAudioSrc:decklinkaudiosrc0: Internal data stream error.
>>> Additional debug info:
>>> ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstDecklinkAudioSrc:decklinkaudiosrc0:
>>> streaming stopped, reason not-negotiated (-4)
>>> Execution ended after 0:00:00.093813522
>>> Setting pipeline to NULL ...
>>> Freeing pipeline ...
>>>
>>>
>>> Now I know that the deckling works as I can pass and record audio to a file.
>>> decklinkaudiosrc channels=8 ! audioconvert ! wavenc ! filesink location=recording.wav
>>>
>>> The original test example you sent me does work.
>>> gst-launch-1.0 -v audiotestsrc ! audio/x-raw,channels=8 ! opusenc ! rtpopuspay ! fakesink
>>>
>>> But when I do something similar in my script, I get an error.
>>>
>>> gst-launch-1.0 webrtcbin name=sendonly bundle-policy=3 stun-server=stun://stun.l.google.com:19302 decklinkvideosrc device-number=0 mode=1080p2398 connection=sdi ! videoconvert ! vp9enc target-bitrate=4000000 keyframe-max-dist=24 deadline=1 end-usage=1 cpu-used=8 lag-in-frames=0 ! rtpvp9pay pt=100 ssrc=2 ! queue ! application/x-rtp,media=video,encoding-name=VP9,payload=100 ! sendonly. decklinkaudiosrc channels=8 ! opusenc ! rtpopuspay ! fakesink
>>>
>>> Setting pipeline to PAUSED ...
>>> Pipeline is live and does not need PREROLL ...
>>> Pipeline is PREROLLED ...
>>> Setting pipeline to PLAYING ...
>>> New clock: GstSystemClock
>>> Redistribute latency...
>>> ERROR: from element /GstPipeline:pipeline0/GstDecklinkAudioSrc:decklinkaudiosrc0: Internal data stream error.
>>> Additional debug info:
>>> ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstDecklinkAudioSrc:decklinkaudiosrc0:
>>> streaming stopped, reason not-negotiated (-4)
>>> Execution ended after 0:00:00.072949718
>>> Setting pipeline to NULL ...
>>> Freeing pipeline ...
>>>
>>>
>>> What was the source your team used for testing multiopus?
>>>
>>> Thanks,
>>>
>>> Patrick
>>>
>>>> On Mar 27, 2022, at 9:13 AM, Nirbheek Chauhan via gstreamer-devel <gstreamer-devel_at_lists_freedesktop_org_59mnm7srbr1196_2a6d20b7 at icloud.com> wrote:
>>>>
>>>> On Sat, Mar 26, 2022 at 5:23 AM Patrick Cusack <patrickcusack at mac.com> wrote:
>>>>> That failed as well. I am using 1.16.2 (which I understand is not compatible with multiopus). I am going to build a Fedora or Debian image and try that using 1.20. Once I get the 8 channels working, is there a way to specify just 6 of the 8 channels into the opusenc? Would that be audio/x-raw,channels=6 ?
>>>>>
>>>>
>>>> If you want to capture 6 channels, you should configure
>>>> decklinkaudiosrc to output 6 channels.
>>>>
>>>> If you want to convert 8 channels to 6 channels, you can use elements
>>>> like audioconvert or audiomixmatri to do that.
>>>>
>>>> Cheers,
>>>> Nirbheek
>>>
>>
>
More information about the gstreamer-devel
mailing list