Problem with NDISRC, WHIPSINK, and audio

GST Developer gstreamer at gallery.co.uk
Sat Jul 15 08:30:05 UTC 2023 

Thanks for your reply.

Sadly, this doesn't change the behaviour - when attempting to send video *and* audio, it stalls after “Redistribute latency…”

So, Just video works:
gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !   whip.  whipsink name=whip   auth-token=“…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”


and If I try *just* audio:
gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.audio ! queue ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'  !  whip.  whipsink name=whip   auth-token=“...redacted..."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
This also works !

but when I try to combine them : 

gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !  whip.  demux.audio ! queue ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'  !  whipsink name=whip   auth-token="…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”

it stalls after “Redistribute latency…”

This does feel like some sort of demux deadlock between the ndisrcdemux and the whipsink.  I have pasted the debug output for the stall loop below, which repeats over and over:

0:00:19.425157202  9470 0x557d29708a00 TRACE              structure gststructure.c:292:gst_structure_new_id_empty_with_size: created structure 0x557d298c1aa0
0:00:19.425181034  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'long-name'
0:00:19.433174494  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
0:00:19.433194340  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'klass'
0:00:19.433204782  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
0:00:19.433216613  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'description'
0:00:19.433223154  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
0:00:19.433232379  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'author'
0:00:19.433246113  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
0:00:19.433254115  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:627:gst_registry_chunks_load_feature: Element factory : npadtemplates=2
0:00:19.433261379  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:526:gst_registry_chunks_load_pad_template: Reading/casting for GstRegistryChunkPadTemplate at address 0x7f3208f72f30
0:00:19.433271543  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:540:gst_registry_chunks_load_pad_template: Added pad_template src
0:00:19.433279923  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:526:gst_registry_chunks_load_pad_template: Reading/casting for GstRegistryChunkPadTemplate at address 0x7f3208f72fb0
0:00:19.433289253  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:540:gst_registry_chunks_load_pad_template: Added pad_template sink
0:00:19.433302373  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistry.c:592:gst_registry_add_feature:<registry0> adding feature 0x557d298bf6d0 (avdec_wmavoice)
0:00:19.433311993  9470 0x557d29708a00 DEBUG        GST_REFCOUNTING gstobject.c:708:gst_object_set_parent:<avdec_wmavoice> set parent (ref and sink)
0:00:19.433322596  9470 0x557d29708a00 TRACE        GST_REFCOUNTING gstobject.c:292:gst_object_ref_sink:<avdec_wmavoice> 0x557d298bf6d0 ref_sink 1->2
0:00:19.433330585  9470 0x557d29708a00 LOG             GST_REGISTRY gstregistry.c:610:gst_registry_add_feature:<registry0> emitting feature-added for avdec_wmavoice
0:00:19.433341108  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:741:gst_registry_chunks_load_feature: Added feature avdec_wmavoice, plugin 0x557d298724d0 libav
0:00:19.433350373  9470 0x557d29708a00 DEBUG           GST_REGISTRY gstregistrychunks.c:583:gst_registry_chunks_load_feature: Plugin 'libav' feature 'avdec_ws_snd1' typename : 'GstElementFactory'
0:00:19.433362464  9470 0x557d29708a00 TRACE        GST_REFCOUNTING gstobject.c:208:gst_object_init:<GstObject at 0x557d298c2050> 0x557d298c2050 new
0:00:19.433374983  9470 0x557d29708a00 LOG             GST_REGISTRY gstregistrychunks.c:610:gst_registry_chunks_load_feature: Reading/casting for GstRegistryChunkElementFactory at address 0x7f3208f72ff0
0:00:19.433385348  9470 0x557d29708a00 TRACE              structure gststructure.c:292:gst_structure_new_id_empty_with_size: created structure 0x557d298c1e40
0:00:19.433394095  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'long-name'
0:00:19.433402420  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'
0:00:19.433411420  9470 0x557d29708a00 DEBUG              structure gststructure.c:2258:gst_structure_parse_field: trying field name 'klass'
0:00:19.433419783  9470 0x557d29708a00 DEBUG                default gstvalue.c:2776:_priv_gst_value_parse_value: trying type name 'string'




> On 15 Jul 2023, at 01:17, Tarun Tej K <tarun4690 at gmail.com> wrote:
> 
> Hi,
> 
> What is the idea behind fixing the sink pad number i.e., sink_0 to video and sink_1 to audio ? The pads would be linked dynamically anyway.
> 
> Can you try the below modified pipeline. I have also added a queue in audio branch after demux.
> 
> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !  whip. demux.audio ! queue ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'  !  whipsink name=whip   auth-token="…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
> 
> 
> On Sat, 15 Jul, 2023, 03:30 GST Developer via gstreamer-devel, <gstreamer-devel at lists.freedesktop.org <mailto:gstreamer-devel at lists.freedesktop.org>> wrote:
>> Hi Folks.
>> 
>> I am attempting use gstreamer to send content from an NDI Source to a WHIP end point at Dolby.io.
>> 
>> If I just send VIDEO, its working fine:
>> 
>> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !   whip.sink_0 whipsink name=whip   auth-token=“…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>> 
>> This works FINE !! we get the signal to the WHIP server and the end to end latency is about 2 seconds.
>> 
>> Now I want to add AUDIO to that pipeline, and I am trying: 
>> 
>> gst-launch-1.0 ndisrc ndi-name="NDIPE8 (SIGGEN)" !  ndisrcdemux name=demux demux.video ! queue ! videoconvert ! x264enc ! video/x-h264,format=byte-stream,profile=baseline ! rtph264pay !   'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' !  whip.sink_0 demux.audio ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2'  ! whip.sink_1  whipsink name=whip   auth-token="…redacted….."  whip-endpoint="https://director.millicast.com/api/whip/myStreamName”
>> 
>> Now, it starts up with:
>> 
>> Setting pipeline to PAUSED ...
>> Pipeline is live and does not need PREROLL ...
>> Pipeline is PREROLLED ...
>> Setting pipeline to PLAYING ...
>> New clock: GstSystemClock
>> Redistribute latency...
>> Redistribute latency...
>> Redistribute latency…
>> 
>> But it stalls right there, and never gets to counting time, and we dont see anything arrive at the WHIP server.
>> 
>> Might anyone know what I am doing wrong ?
>> 
>> Many thanks !!!
>> 
>> 
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230715/c07f3323/attachment.htm>

More information about the gstreamer-devel mailing list