How to sync movie and audio with rtpbin in gstremer

[SHIOKAWA TSUYOSHI(塩川 健)]

Hi,
I would like to know how to synchronize H264 movie and G711 audio on the WebRTC server with pipeline by rtpbin.
I have tried this parameter several times and didn't work, please let me know what is wrong in this parameter.
Let me know if this doesn't makes sense for you, I will explain more.
Can you advise me ?

<Background>
I use gstreamer and janus on the same server.
A movie data and audio data are sent to gstreamer with rtp.
After then I use <No.1 pipeline> to synchronize movie and audio with rtpbin and pass them to janus.
rtpbin will synchronize the movie and audio by each timestamp of data in my understanding.
But I found that they were NOT sync.
It meant that the movie and audio were out of sync.
I am trying to find out where the problem is and I am thinking of using <No.2 pipeline> between rtp and rtpbin to cause the situation of out of sync.
And if out of sync situation happens, then that means there is a problem with pipeline.
 However, I am also not sure about this <No.2 pipeline> to make it work, so I would like to make sure if my understanding is correct.


<No.1 Pipeline>
gst-launch-1.0 rtpbin name=rtpbin rtpbin name=rtpbin2
udpsrc port=60000 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)MP2T-ES" !
rtpbin.send_rtp_sink_0
rtpbin.send_rtp_src_0 ! rtpmp2tdepay ! queue max-size-time=0 ! tsdemux !
video/x-h264,framerate=30/1,width=1280,height=960,tune=zerolatency !
h264parse ! rtph264pay ! rtpbin2.send_rtp_sink_0
udpsrc port=60001 ! rtpbin.recv_rtcp_sink_0
rtpbin.send_rtcp_src_0 ! udpsink host=192.168.31.1 port=60011 sync=false async=false
rtpbin2.send_rtp_src_0 ! udpsink host=127.0.0.1 port=5005
rtpbin2.send_rtcp_src_0 ! udpsink host=127.0.0.1 port=5105 sync=false async=false
udpsrc port=51005 ! rtpbin2.recv_rtcp_sink_0

udpsrc port=61002 caps="application/x-rtp, media=(string)audio, clock-rate=(int)8000, encoding-name=(string)PCMU" !
rtpbin.send_rtp_sink_1
rtpbin.send_rtp_src_1 ! queue ! rtppcmudepay ! rtppcmupay ! rtpbin2.send_rtp_sink_1

udpsrc port=60003 ! rtpbin.recv_rtcp_sink_1
rtpbin.send_rtcp_src_1 ! udpsink host=192.168.31.1 port=60013 sync=false async=false

rtpbin2.send_rtp_src_1 ! udpsink host=127.0.0.1 port=5003
rtpbin2.send_rtcp_src_1 ! udpsink host=127.0.0.1 port=5103 sync=false async=false
udpsrc port=51003 ! rtpbin2.recv_rtcp_sink_1


<No.2 Pipeline>
gst-launch-1.0 udpsrc port=61002 ! queue max-size-time=0 min-threshold-time=1500000000 ! udpsink host=192.168.31.138 port=60002


Best regards.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20200715/b1d79f2b/attachment-0001.htm>