Inconsistent/variable delay between USB and onboard alsasrc

[Nicolas Dufresne]

Le mercredi 21 juillet 2021 à 16:58 +0200, Michiel Konstapel via gstreamer-devel
a écrit :
> I am recording audio from two alsasrc elements: an analog line input 
> from the onboard sound card, and a USB audio device (USB microphone). I 
> need the recordings to be synchronized, so playing them back together 
> doesn't produce a nasty "tiled bathroom" sound. I expected the different 
> sources to have different internal delays, which I can adjust for with a 
> probe that adjusts the buffer timestamp.
> 
> However, there seems to be no consistency in the time difference: in one 
> run, the analog input is 3 ms behind the USB audio, the next time it's 
> 44 ms, then it's 20 ms... I measured this by placing the microphones 
> side by side, producing a clap or tick and comparing the tracks in Audacity.
> 
> I've tried
> - always choosing the onboard card as the clock source
> - choosing neither and using the system clock
> - all the options for alsasrc slave-method except "none"
> 
> How should I set things up so the sources are reliably synchronized?

This one is not as much about slaving as it is about getting HW timestamp. First
step, make sure your driver supports that. Run your pipeline with GST_DEBUG=4
(info level), and looks for that trace:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/ext/alsa/gstalsasrc.c#L276

Then monitor for:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/ext/alsa/gstalsasrc.c#L993

This will first ensure that you have timestamp that aligns more closely to
reality. Then the synchronization does not depends on the src, but on the sink
element. By default, this is set to slave-method "skew" with alignment-threshold
of 40ms. Meanwhile even if you have non-drifting audio, it will simply break the
stream to fix it once the delay falls passed this threshold. That makes your
observation very consistent, since with a margin of error, your result ranges
within 40ms, without drift, it will stay at that position.

skew is pretty much the only working option in alsasink atm, play with
alignment-threshold and drift-tolerance in order to get something closer to 4ms.

An alternative is to mix the streams inside gstreamer, but that introduce a lot
of latency. audiomixer will do a better job at aligning the streams.

> 
> Kind regards,
> Michiel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel