audiomixer output is delayed silence when 2nd sink pad is added

[Pradeep Acharya]

Hi,

I;m using audiomixer element to mix audio from live sources in a video
conference call. The o/p of audio mixer is given to the opus encoder and
then transmitted to the remote end . It works fine when there is only one
one sink pad attached to the mixer i,,e only one client joins the
conference. Problem starts when the 2nd participant joins and a second sink
pad is added to the mixer.

I observe a delay proportional to the time the first user joined. After
some time, the mixer starts to output data. Audio is not lost but delayed
and the delay continues. I tried some of the configuration parameters like
" start-time-selection" to 1 and "start-time" to 0. ,
"ignore-inactive-pads" to TRUE. There is no improvement even after
configuring these params. From gstreamer logs, i could see that

*0:00:39.011285060* ^[[32m3313765^[[00m 0x7f6fdc00e460 ^[[37mDEBUG  ^[[00m
^[[00;35m          aggregator
gstaggregator.c:2172:gst_aggregator_query_latency_unlocked:<audiomixer_111>^[[00m
configured
*latency live:true min:20000000 max:20000000  ---> when first sink pad is
added *

0:00:39.022481190 ^[[32m3313765^[[00m 0x*7f7068024760* ^[[33;01mLOG
 ^[[00m ^[[00;35m          aggregator
gstaggregator.c:3415:gst_aggregator_pad_buffer_consumed:<audiomixer_111:sink_0>^[[00m
Signaling buffer consumed from thread 0x7f7068024760
0:00:39.022543769 ^[[32m3313765^[[00m 0x*7f706802476*0 ^[[37mDEBUG  ^[[00m
^[[00;35m          aggregator
gstaggregator.c:3513:gst_aggregator_pad_pop_buffer:<audiomixer_111:sink_0>^[[00m
Consumed: buffer: 0x7f6fa0004120, pts 0:00:00.822554913, dts
0:00:37.845570590, dur 0:00:00.020000000, size 3840, offset none,
offset_end none, flags 0x0

0:00:39.023672822 ^[[32m3313765^[[00m 0x*7f7068024760* ^[[37mTRACE  ^[[00m
^[[00;35m          aggregator
gstaggregator.c:3446:gst_aggregator_pad_clip_buffer_unlocked:<audiomixer_111:sink_0>^[[00m
Clipping: buffer: 0x7f6fa0004240, pts 0:00:00.842553968, dts
0:00:37.847505654, dur 0:00:00.020000000, size 3840, offset none,
offset_end none, flags 0x800

0:00:39.023926552 ^[[32m3313765^[[00m 0x7f7068024760 ^[[37mTRACE  ^[[00m
^[[00;35m          aggregator
gstaggregator.c:713:gst_aggregator_default_finish_buffer:<audiomixer_111>^[[00m
GstAggregator: 0x7f70680562a0 *pushing buffer buffer: *0x7f6ff4064900, pts
0:00:00.842541666, dts 99:99:99.999999999, dur 0:00:00.020000000, size
3840, offset 40442, offset_end 41402, flags 0x800


When 2nd participant joins after 8 minutes, sink_1 pad is created for
audiomixer.  Audio o/p is delayed by ~ 38 seconds i,,e time duration of
1st sink pad's *(sink_0 )* buffer start time .

*0:08:53.918540128* ^[[32m3313765^[[00m 0x*7f6f98006e70* ^[[36mINFO
^[[00m ^[[00;35m          aggregator
gstaggregator.c:1524:gst_aggregator_start_srcpad_task:<audiomixer_111>^[[00m
Starting srcpad task   ---> when second sink pad is created sink_1
*0:08:53.918829275* ^[[32m3313765^[[00m 0x*7f6fdc04a700* ^[[33;01mLOG
 ^[[00m ^[[00;35m          aggregator
gstaggregator.c:1380:gst_aggregator_aggregate_func:<audiomixer_111>^[[00m
Checking aggregate

I see errors like pads not ready : no sink pads. However 2 sink pads were
already created
0:10:17.737484397 ^[[32m3313765^[[00m 0x*7f7068024760* ^[[33;01mLOG
 ^[[00m ^[[00;35m          aggregator
gstaggregator.c:567:gst_aggregator_check_pads_ready:<audiomixer_111>^[[00m
pads not ready: no sink pads
0:10:17.737527246 ^[[32m3313765^[[00m 0x*7f7068024760* ^[[37mDEBUG  ^[[00m
^[[00;35m          aggregator
gstaggregator.c:893:gst_aggregator_wait_and_check:<audiomixer_111>^[[00m
*possibly
waiting for clock to reac*h 1246:02:12.632613629 (base 1245:51:56.05007196*3
start 0:10:16.562541666* latency 0:00:00.020000000 current
1246:02:12.613770784)


*The source code works fine if i replace audio mixer element  with adder
element  without modifying any other configurations or elements of the
pipeline. This makes me to suspect audiomixer.  i don't see delay issues
that i see using the audiomixer. Any suggestions on how to debug and what
to look for ? Should i continue using adder element instead of audiomixer. *

Thanks & Regards
Pradeep
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20221223/1d2fb21c/attachment.htm>