Audio gets out of sync after a 2 minutes in webrtc session

[Carlos Yuste]

Hello all and happy new year !

I have recently started creating gstreamer pipelines, so apologies if I'm
missing something obvious.
I am trying to configure a raspberry pi 4 with the v1.2 camera and a cheap
USB microphone to act as a webrtc peer. In my scenario, the raspberry will
be the only peer sending audio/video, the other peer will only act only as
receiver.

To integrate webrtc and gstreamer I'm using this example as base:
https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/blob/master/samples/kvsWebRTCClientMasterGstreamerSample.c
I don't mind sharing the actual code I'm using but I think it will be
easier to reproduce this way.

I have tested several configurations for the pipeline; my main goal is to
achieve the lowest latency possible with a reasonable bandwidth, I don't
mind having a low-quality output.

Currently, the best I have managed to configure is this:

"v4l2src do-timestamp=TRUE device=\"/dev/video0\" ! videoflip
method=rotate-180 ! queue ! videoconvert ! "
"video/x-raw,format=I420,width=640,height=480,framerate=25/1 ! "
"omxh264enc control-rate=1 target-bitrate=500000 periodicty-idr=25
inline-header=FALSE ! "
"h264parse config-interval=-1 ! "
"video/x-h264,stream-format=byte-stream,alignment=au,width=640,height=480,framerate=25/1,profile=baseline
! "
"appsink sync=TRUE emit-signals=TRUE name=appsink-video "
"alsasrc use-driver-timestamps=FALSE device=plughw:1 ! queue
max-size-buffers=1 ! opusenc ! "
"audio/x-opus ! appsink max-buffers=2 drop=true sync=TRUE emit-signals=TRUE
name=appsink-audio"

With this pipeline, audio and video start synchronized and with low latency
but after a while, around 2 minutes, "something" happens and audio starts
getting more and more out of sync with the video. Video keeps showing with
low latency in all my tests with this configuration. In the chrome webrtc
stats it's easy to see when the audio lag starts to degrade (blue arrow)
[image: image.png]

 I don't know exactly what is the problem, these tests have been executed
in my office, with good WiFi coverage and without using TURN. When playing
the CPU cores are below 25% so it doesn't look to be overloaded either

Thank you very much,
Carlos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230102/6422dca2/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 247361 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230102/6422dca2/attachment-0001.png>