problem with gstreamer compositor

[Nicolas Dufresne]

Hi,

Le mar. 27 déc. 2022, 06 h 15, Masoud MJ via gstreamer-devel <
gstreamer-devel at lists.freedesktop.org> a écrit :

> Hi,
>
> I'm currently writing a webrtc mcu using rust and gstreamer.
> I get video tracks from webrtc peer using webrtc-rs crate and forward them
> to gstreamer
> and then I will arrange them in grid layout form using compositor element
> and then get output using appsink element and then forward that to other
> webrtc peer.
>
> The project has three parts :
>
> 1- front
> 2- main server
> 3- compositor server
>
> both servers are written in rust
>
> The problem is when I connect front part directly to compositor server it
> totally works well but if front connects to main server and then main
> server connects to compositor server
> the video from compositor server will arrive to browser almost at 2-3
> minutes later ...
>
> I guess it a timing or clock problem
>


> I mention some of my debugging results as hints :
>
> 1-I used udpsrc instead of appsrc and did not have any effect
>

In both cases, your apps need to enable do-timestamp, as RTP stack needs
the arrival time. appsrc needs is-live to be set. Not timestamp and not
live sources can cause issues, not sure if that would fully match the
symptoms.



> 2-I used single appsrc with ssrcdemux element instead of multiple appsrc
> and did not have any effect
>
> 3-I used decodebin instead of rtph264depay + avdec_h264 and did not have
> any effect
>
> 4-I implemented main server in both Go and Rust and behaviour of them was
> same
>
> 5-I tested webrtc-rs in both single port and multi port and both showed
> same behaviour
>
> 6-Packeting rtp with gstreamer rtph264pay or in webrtc-rs also have same
> behaviour
>
> I created a minimal as possible copy of my project for your test.
>

During holidays, i stay away from this, but I'd verify if the delay matches
exactly the time you have been playing through before adding something.

If so, it likely be that the start time mode (sorry can't remember the
property name) isn't set correctly on compositor. The compositor, will by
default fill the portion of time from 0 to now with black images.


> ------------------------------------------------------------
>
> Test steps for direct connection from browser to compositor server :
>
> 1-in "compositor server" folder run "cargo run"
>
> 2-in "front" folder edit index.html file and set "SERVER_PORT" variable to
> 8080
>
> 3-open index.html in browser
>
> 4-click on Add Webcam button as many times as you want
>
>
> Test steps for connection from browser to main server and then from main
> server to compositor server :
>
> 1-in "compositor server" folder run "cargo run"
>
> 2-in "front" folder edit index.html file and set "SERVER_PORT" variable to
> 9090
>
> 3-in "main server" folder run "cargo run"
>
> 4-open index.html in browser
>
> 5-click on Add Webcam button as many times as you want
>
> ------------------------------------------------------------
> Thank you in advance.
>  gst-compositor.zip
> <https://drive.google.com/file/d/14sefW_L9lFnl_d__D_AZTVXqoqB29i_Z/view?usp=drive_web>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20221228/8f386ff3/attachment.htm>