<html><head></head><body><div>Hi,</div><div><br></div><div>One thing you can do is enable qos=true on both the sink (to generate the events). An then enable it also on the encoder to drop frames before encoding to catch up.</div><div><br></div><div>Olivier</div><div><br></div><div>On Fri, 2021-04-16 at 09:43 +0200, Guillaume Denis wrote:</div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div>Hello,<br></div><div><br></div><div>I am encountering an issue with a server side processing of WebRTC video <br></div><div>streams: an added/variable latency appears and increases over time (but <br></div><div>I don't notice any obvious "slow motion" effect).<br></div><div><br></div><div>The pipeline is defined as:<br></div><div>appsrc format=time is-live=true do-timestamp=true name=src ! <br></div><div>application/x-rtp, encoding-name=VP8-DRAFT-IETF-01 ! queue ! rtpvp8depay <br></div><div>! decodebin ! videoconvert ! identity ! videoconvert ! vp8enc <br></div><div>error-resilient=partitions keyframe-max-dist=10 auto-alt-ref=true <br></div><div>cpu-used=5 deadline=1 ! rtpvp8pay ! appsink name=sink<br></div><div><br></div><div>(the intent is to replace identity by other effects)<br></div><div><br></div><div>The processing is done in parallel on two video streams:<br></div><div>peer A webcam from browser -> webrtc app -> GStreamer pipeline -> webrtc <br></div><div>app -> peer B screen in browser<br></div><div>peer B webcam from browser -> webrtc app -> GStreamer pipeline -> webrtc <br></div><div>app -> peer A screen in browser<br></div><div><br></div><div>The webrtc app is made with the pion/webrtc library (for <br></div><div>signaling/connecting/routing of streams...), and an additional <br></div><div>processing is done (with GStreamer) on audio streams, without any time <br></div><div>distortion.<br></div><div><br></div><div>I have two ideas:<br></div><div><br></div><div>* either the entire processing (vp8 decoding/encoding to start with) is <br></div><div>expensive enough to "break" real-time and introduce time distortions. I <br></div><div>wonder if in this case there is a way to adapt the processing (for <br></div><div>instance adjusting the framerate) and try to maintain a near constant <br></div><div>latency<br></div><div>* there is a problem with how RTP packets are "time labeled" (with pts <br></div><div>for instance) and synced<br></div><div><br></div><div>I would greatly appreciate any help, including how to debug/specify more <br></div><div>precisely the encountered behavior and where it occurs.<br></div><div><br></div><div>Many thanks,<br></div><div>Guillaume<br></div><div>_______________________________________________<br></div><div>gstreamer-devel mailing list<br></div><div><a href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a><br></div><div><a href="https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a><br></div></blockquote><div><br></div><div><span><pre>-- <br></pre><div data-evo-paragraph="" class="">Olivier Crête</div><div data-evo-paragraph="" class=""><a href="mailto:olivier.crete@collabora.com">olivier.crete@collabora.com</a></div><div data-evo-paragraph="" class=""><br></div></span></div></body></html>