rtpbin: ts adjustment needs filtering/smoothing

[charleslaub at sbcglobal.net]

I think I have raised a "false flag" about the source of the audio problems I am experiencing. 

I am not convinced the problem lies with resampling in the alsasink/GstAudioSink because when I set the alsasink property slave-method=none the ticks and pops continue. The only other place in the pipeline (from what I understand!) where timing is being adjusted is in the rtpjitterbuffer. I probably should have made that more clear in my original post. 

I am using rtpbin, which is a wrapper for several elements including rtpjitterbuffer. Initially I used the default value of max-ts-offset-adjustment, which is "0" but I think means "no limit". Then I measured the playback timing difference between two clients and it was moving around by several (occasionally more than 10) milliseconds every few seconds! This is very audible. Next I tried to limit the ts adjustments using max-ts-offset-adjustment, but I found I had to set it all the way down to 1 or 2 to minimize the changes made to ts. Thy timing difference was much more stable but still moved around sightly in my measurements. I think this may be why I hear ticks and pops in the audio playback.

At that point I thought that perhaps smoothing out the RTCP sender report clock/timing info might be one way to address this problem, if that is even what is going on. Since the "by eye average" playback timing seems about right, smoothing out the short term jitter seems to make sense.  

-Charlie


-----Original Message-----
From: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> On Behalf Of Sebastian Dröge
Sent: Saturday, October 17, 2020 4:49 PM
To: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Subject: Re: rtpbin: ts adjustment needs filtering/smoothing

On Sat, 2020-10-17 at 13:13 -0400, Nicolas Dufresne wrote:
> 
> This is pure C, there is no ALSA there. It is about 50 line of code.
> We have another resampler now, but no one looked it up to see if it 
> can be used more dynamicly then how it is used in audioresample 
> element.

I actually looked at that a while ago and came to the conclusion that the resampler is fine and could be used for this, but the way how GstAudioBaseSink and GstAudioRingBuffer work does not really fit with how a resampler would behave. This needs some redesigning of how these work together.

Apart from the current resampler in there being bad (it's just linear interpolation), this is also a big reason why it doesn't work at all currently.

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel