Time distortion on WebRTC video streams

[Olivier Crête]

Hi,

One thing you can do is enable qos=true on both the sink (to generate
the events). An then enable it also on the encoder to drop frames
before encoding to catch up.

Olivier

On Fri, 2021-04-16 at 09:43 +0200, Guillaume Denis wrote:
> Hello,
> 
> I am encountering an issue with a server side processing of WebRTC
> video 
> streams: an added/variable latency appears and increases over time
> (but 
> I don't notice any obvious "slow motion" effect).
> 
> The pipeline is defined as:
> appsrc format=time is-live=true do-timestamp=true name=src ! 
> application/x-rtp, encoding-name=VP8-DRAFT-IETF-01 ! queue !
> rtpvp8depay 
> ! decodebin ! videoconvert ! identity ! videoconvert ! vp8enc 
> error-resilient=partitions keyframe-max-dist=10 auto-alt-ref=true 
> cpu-used=5 deadline=1 ! rtpvp8pay ! appsink name=sink
> 
> (the intent is to replace identity by other effects)
> 
> The processing is done in parallel on two video streams:
> peer A webcam from browser -> webrtc app -> GStreamer pipeline ->
> webrtc 
> app -> peer B screen in browser
> peer B webcam from browser -> webrtc app -> GStreamer pipeline ->
> webrtc 
> app -> peer A screen in browser
> 
> The webrtc app is made with the pion/webrtc library (for 
> signaling/connecting/routing of streams...), and an additional 
> processing is done (with GStreamer) on audio streams, without any
> time 
> distortion.
> 
> I have two ideas:
> 
> * either the entire processing (vp8 decoding/encoding to start with)
> is 
> expensive enough to "break" real-time and introduce time distortions.
> I 
> wonder if in this case there is a way to adapt the processing (for 
> instance adjusting the framerate) and try to maintain a near constant
> latency
> * there is a problem with how RTP packets are "time labeled" (with
> pts 
> for instance) and synced
> 
> I would greatly appreciate any help, including how to debug/specify
> more 
> precisely the encountered behavior and where it occurs.
> 
> Many thanks,
> Guillaume
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-- 
Olivier Crête
olivier.crete at collabora.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210416/60c06340/attachment.htm>