webrtcbin no videostream when source is RTP

[Peter Biro]

Hi all,

I see a very strange issue and I cannot crack it for a while.

In our application we are using multiple pipelines the first one is the source which sends the camera feed as h264 RTP stream to processors:

gst-launch-1.0 -e\
 nvarguscamerasrc sensor-id="$SENSOR_ID" sensor-mode=0 gainrange="1 16" ispdigitalgainrange="1 1" name="${APP_NAME}_pipeline_overrides_${SENSOR_ID}"\
    ! "video/x-raw(memory:NVMM), width=(int)${capture_width}, height=(int)${capture_height}, format=(string)NV12, framerate=(fraction)${capture_framerate}/1"\
    ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name="$customer_lib" ! 'video/x-raw(memory:NVMM), format=(string)NV12'\
    ! nvvidconv ! omxh264enc bitrate="${capture_bitrate}" control-rate=1 ! rtph264pay mtu=1400\
    ! udpsink auto-multicast=true clients="${udp_clients}"

On the receiving side (one of the receivers) would forward it to browsers via WebRTC for this Im using the following pipeline (signaling is carried out with a simple c++ application wrapping this):

udpsrc multicast-group=<DCL_WEBRTC_UDP_SOURCE_IP> auto-multicast=true port=<DCL_WEBRTC_UDP_SOURCE_PORT>
       ! application/x-rtp,media=video,encoding-name=H264,payload=96,clock-rate=90000,packetization-mode=1,profile-level-id=424015,sprop-parameter-sets="Z0JAKJWgHgCJ+VA=,aM48gA==",src=1728323247,timestamp-offset=2499875162,seqnum-offset=11758,a-framerate=20
       ! webrtcbin name=webrtc_sink stun-server='stun://localhost:3478' latency=0

in the browser I can see that the stream is connected and using chrome://webrtc-internals/ <chrome://webrtc-internals/> I can see that data is coming in but the video not showing up

I was able to receive video in the browser if I had the webrtc node and the camerasource in one pipeline, so I was sure that the issue is not with the H262. I was fooling around with the pipelines and it turend out if I decode and encode the stream again the video showed up again in the browser.

udpsrc multicast-group=<DCL_WEBRTC_UDP_SOURCE_IP> auto-multicast=true port=<DCL_WEBRTC_UDP_SOURCE_PORT>
        ! application/x-rtp,media=video,encoding-name=H264,payload=96,clock-rate=90000,packetization-mode=1,profile-level-id=424015,sprop-parameter-sets="Z0JAKJWgHgCJ+VA=,aM48gA==",src=1728323247,timestamp-offset=2499875162,seqnum-offset=11758,a-framerate=20
        ! rtpjitterbuffer ! rtph264depay ! h264parse
        ! omxh264dec ! videoconvert
        ! omxh264enc bitrate=6000000 control-rate=1 insert-sps-pps=true ! rtph264pay mtu=1400 
        ! webrtcbin name=webrtc_sink stun-server='stun://localhost:3478' latency=0

My best suspect is that there are a few differences in the SDPs (I highlighted the differences), but this maybe a wrong track:

"good" case:
v=0
o=- 4683413980427089901 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=msid-semantic:WMS webrtc_sink
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:ldIonoD8h44Onzh25CF+WghE9PF4aydg
a=ice-pwd:VvGX0RUQLHM/UexT6iDYWnus2dvDpId3
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=fmtp:96 packetization-mode=1;profile-level-id=424015;sprop-parameter-sets=Z0JAKJWgHgCJ+VA=,aM48gA==
a=ssrc:1591034174 msid:user3742382827 at host-d73d2844 webrtctransceiver0
a=ssrc:1591034174 cname:user3742382827 at host-d73d2844
a=mid:video0
a=fingerprint:sha-256 3A:EC:E2:B8:A4:D2:72:A3:D5:47:45:D8:C5:E4:0F:87:69:3D:33:2A:A9:38:1B:96:E1:D9:FC:81:D0:AF:2E:10

"wrong" case:
v=0
o=- 2325516305835473545 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=msid-semantic:WMS webrtc_sink
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:L48buQzgWBf0WEM2iGR9lPVnYI85MW0D
a=ice-pwd:3VeswKVaJ29Xhu/4fJo4MlDnl3scYKpP
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=fmtp:96 sprop-parameter-sets=Z0JAKJWgHgCJ+VA=,aM48gA==
a=mid:video0
a=fingerprint:sha-256 A1:06:98:BB:19:95:67:40:0E:A0:38:EA:7C:D0:A5:9D:8C:7C:D6:5B:73:A1:CE:08:6B:1F:4B:26:1B:22:6B:55


Do you have any idea what I am doing wrong here? What is Im missing here? 


Thanks!

Bests,
Peter


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20210524/193cd79d/attachment-0001.htm>