problem with gstreamer compositor

[Masoud MJ]

Hi,

I'm currently writing a webrtc mcu using rust and gstreamer.
I get video tracks from webrtc peer using webrtc-rs crate and forward them
to gstreamer
and then I will arrange them in grid layout form using compositor element
and then get output using appsink element and then forward that to other
webrtc peer.

The project has three parts :

1- front
2- main server
3- compositor server

both servers are written in rust

The problem is when I connect front part directly to compositor server it
totally works well but if front connects to main server and then main
server connects to compositor server
the video from compositor server will arrive to browser almost at 2-3
minutes later ...

I guess it a timing or clock problem

I mention some of my debugging results as hints :

1-I used udpsrc instead of appsrc and did not have any effect

2-I used single appsrc with ssrcdemux element instead of multiple appsrc
and did not have any effect

3-I used decodebin instead of rtph264depay + avdec_h264 and did not have
any effect

4-I implemented main server in both Go and Rust and behaviour of them was
same

5-I tested webrtc-rs in both single port and multi port and both showed
same behaviour

6-Packeting rtp with gstreamer rtph264pay or in webrtc-rs also have same
behaviour

I created a minimal as possible copy of my project for your test.

------------------------------------------------------------

Test steps for direct connection from browser to compositor server :

1-in "compositor server" folder run "cargo run"

2-in "front" folder edit index.html file and set "SERVER_PORT" variable to
8080

3-open index.html in browser

4-click on Add Webcam button as many times as you want


Test steps for connection from browser to main server and then from main
server to compositor server :

1-in "compositor server" folder run "cargo run"

2-in "front" folder edit index.html file and set "SERVER_PORT" variable to
9090

3-in "main server" folder run "cargo run"

4-open index.html in browser

5-click on Add Webcam button as many times as you want

------------------------------------------------------------
Thank you in advance.
 gst-compositor.zip
<https://drive.google.com/file/d/14sefW_L9lFnl_d__D_AZTVXqoqB29i_Z/view?usp=drive_web>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20221227/185f4911/attachment.htm>