Using GStreamer for audio and video synchronization will have a delay of about one second

[Nirbheek Chauhan]

Hi,

You have a delay of one second because that's the total latency of the
elements in the pipeline. By disabling sync, you are telling gstreamer
to ignore that latency.

Specifically, you have a 700ms latency in tsdemux by default, and
you've set a jitterbuffer size of 200ms, totalling 900ms. You can try
reducing tsdemux latency by setting the corresponding property. Since
you are actually using RTP for timing and not MPEG-TS, you can
probably also set ignore-pcr=true.

Further reading:
https://gstreamer.freedesktop.org/documentation/application-development/advanced/clocks.html
https://gstreamer.freedesktop.org/documentation/additional/design/latency.html

Cheers,
Nirbheek

On Fri, Dec 3, 2021 at 4:30 PM 吴高升 via gstreamer-devel
<gstreamer-devel at lists.freedesktop.org> wrote:
>
>  I use GStreamer to play UDP video stream, and the streaming end is the screen projection function in Android phones.My GStreamer pipeline is as follows:
>
> ```
> udpsrc port=1991 caps="application/x-rtp,clock-rate=90000, media=video" ! rtpjitterbuffer latency=200  mode=4 ! rtpmp2tdepay ! tsdemux name=demuxer demuxer. ! queue name=video_ch max-size-buffers=0 max-size-time=0 ! h264parse ! queue name=dec0 max-size-buffers=0 max-size-time=0 ! avdec_h264 max-threads=0 skip-frame=1 output-corrupt=false ! videoconvert n-threads=0 ! autovideosink force-aspect-ratio=false sync=false async=false demuxer. ! queue max-size-buffers=0 max-size-time=0 ! aacparse ! avdec_aac ! audioconvert ! audioresample ! autoaudiosink sync=false
> ```
>
> However, the audio and video will be out of sync during the work of this pipeline, and the longer the playing time, the more obvious the audio and video out of sync.But there is no delay.When the `sync` parameter settings of `autovideosink` and `autoaudiosink` are deleted, the audio and video can always be synchronized, but there is a delay of about one second.My GStreamer pipeline is as follows:
> ```
> udpsrc port=1991 caps="application/x-rtp,clock-rate=90000, media=video" ! rtpjitterbuffer latency=200  mode=4 ! rtpmp2tdepay ! tsdemux name=demuxer demuxer. ! queue name=video_ch max-size-buffers=0 max-size-time=0 ! h264parse ! queue name=dec0 max-size-buffers=0 max-size-time=0 ! avdec_h264 max-threads=0 skip-frame=1 output-corrupt=false ! videoconvert n-threads=0 ! autovideosink force-aspect-ratio=false async=false demuxer. ! queue max-size-buffers=0 max-size-time=0 ! aacparse ! avdec_aac ! audioconvert ! audioresample ! autoaudiosink
> ```
>
>
> But when using VLC to play network stream directly rtp://@:1991, the delay of video is very small, which is within the range I can accept, and the audio and video can always be synchronized.
>
> I don't know if any of my parameters are set incorrectly, or there is a problem with this audio and video synchronization method.
>
> I urgently need your help.
>
> Thanks!
>
>
>
>
>
>
>
>
>
>
>
>
>
> ----
>
>