H.264 offer from Chrome not negotiable with GStreamer 1.18.4 Python webrtcbin app

Mon Aug 16 10:06:18 UTC 2021

This is the situation:

I'm running a GST 18.4 python app. This app communicates with a Chrome JS app on the same network.

The app is able to send and receive VP8, but I'm having a problem with H.264, in case the Python app (webrtcbin based) is the ANSWERER. The problem is not there if the Python app is the OFFERER.
First the positive case: This is my H.264 pipeline in the Python app. If the Python app OFFERS, the video appears in Chrome.

# H.264 TEST VIDEO
PIPELINE_TEST_H264 = '''
webrtcbin name=webrtcbin bundle-policy=max-bundle stun-server=stun://stun.l.google.com:19302
videotestsrc is-live=true pattern=smpte ! video/x-raw,width=1280,height=720 ! videoconvert ! x264enc ! rtph264pay config-interval=1 ! webrtcbin.
'''

The SDP handshake is like so:

The OFFER sent:

v=0
o=- 143230668495837560 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=group:BUNDLE video0
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:8OgtMvs3Zdn04lGPl61MEeQGtuq2XNS+
a=ice-pwd:GbtIZHC7CWPkgzNrnH7mWEtWNpaeWL5n
a=rtcp-mux
a=rtcp-rsize
a=sendonly
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=framerate:30
a=fmtp:96 packetization-mode=1;profile-level-id=42c01f;sprop-parameter-sets=Z0LAH9kAUAW7AWoCAgKAAAADAIAAAB5HjBkk,aMuMsg==
a=ssrc:2817782937 msid:user2521903457 at host-548ae76e webrtctransceiver0
a=ssrc:2817782937 cname:user2521903457 at host-548ae76e
a=mid:video0
a=fingerprint:sha-256 1B:99:58:8C:F3:33:28:60:02:E6:11:66:74:B7:84:B4:D9:06:32:12:13:23:F0:E9:23:68:9D:C3:CC:79:85:C5
The ANSWER received (video flows):
v=0
o=- 6562431427409268738 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE video0
a=msid-semantic: WMS
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:OoGw
a=ice-pwd:egHkzRQOlN6ADWLnaltQ6AkA
a=ice-options:trickle
a=fingerprint:sha-256 20:C2:D3:AA:37:07:31:87:FA:71:D4:20:CF:50:ED:07:EA:C8:B8:1D:EA:02:73:9C:7C:AC:24:5C:F3:C9:DA:60
a=setup:active
a=mid:video0
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
Works. Video flows.
But now the other way around: The video does not appear in Chrome, if Chrome is the OFFERER. The ANSWER confirms that with "a=inactive".
The OFFER generated by Chrome:
v=0
o=- 6799643414674800834 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 122 102 121 127 120 125 107 108 109 35 36 124 119 123 118 114 115 116 37
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Tyef
a=ice-pwd:tct8Z2AfUb1yNAT/aWDfj2OI
a=ice-options:trickle
a=fingerprint:sha-256 F6:01:90:DF:E5:67:4E:B0:9F:DF:14:1A:3B:08:2F:E8:12:ED:CC:35:EB:4D:E2:22:AF:1F:18:7C:EB:37:D7:41
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:122 VP9/90000
a=rtcp-fb:122 goog-remb
a=rtcp-fb:122 transport-cc
a=rtcp-fb:122 ccm fir
a=rtcp-fb:122 nack
a=rtcp-fb:122 nack pli
a=fmtp:122 profile-id=1
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:121 rtx/90000
a=fmtp:121 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=127
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:35 AV1X/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:124 H264/90000
a=rtcp-fb:124 goog-remb
a=rtcp-fb:124 transport-cc
a=rtcp-fb:124 ccm fir
a=rtcp-fb:124 nack
a=rtcp-fb:124 nack pli
a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032
a=rtpmap:119 rtx/90000
a=fmtp:119 apt=124
a=rtpmap:123 H264/90000
a=rtcp-fb:123 goog-remb
a=rtcp-fb:123 transport-cc
a=rtcp-fb:123 ccm fir
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032
a=rtpmap:118 rtx/90000
a=fmtp:118 apt=123
a=rtpmap:114 red/90000
a=rtpmap:115 rtx/90000
a=fmtp:115 apt=114
a=rtpmap:116 ulpfec/90000
a=rtpmap:37 flexfec-03/90000
a=rtcp-fb:37 goog-remb
a=rtcp-fb:37 transport-cc
a=fmtp:37 repair-window=10000000

The ANSWER generated by webrtcbin:

v=0
o=- 6799643414674800834 2 IN IP4 0.0.0.0
s=-
t=0 0
a=group:BUNDLE 0
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=ice-ufrag:ej11/cl3FCkkhNmNliiql1R5MX5MFn/g
a=ice-pwd:SgutRYS/iWj9l8WvWvXGzrohbIiTLK+f
a=mid:0
a=rtcp-mux
a=setup:active
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 ccm fir
a=inactive
a=fingerprint:sha-256 BD:5C:65:0E:D3:75:67:8B:80:71:57:D5:08:14:70:1A:BB:4F:51:F8:54:53:80:17:8C:3E:44:DB:EA:E5:04:06

I then tried limit the codecs offered by Chrome to one of the seemingly matching H.264 formats by `setCodecPreferences`:

videoTransceiver.setCodecPreferences([
  { clockRate: 90000, mimeType: "video/H264", sdpFmtpLine: "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f" }
])

After this the OFFER from Chrome looks like this:

v=0
o=- 1241534504099527837 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS oQrYP0RdXW3OITfGDlPmkOEuvWzFikm0VwlO
m=video 9 UDP/TLS/RTP/SAVPF 106
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:hO7n
a=ice-pwd:cZ0ULb11MCZp/kiAj1Pq4z7j
a=ice-options:trickle
a=fingerprint:sha-256 9F:2B:04:A2:8C:6F:34:A6:CB:19:20:C5:2E:91:A8:94:94:F8:29:CF:EC:ED:18:DF:19:B8:79:F0:6A:E4:4A:64
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendrecv
a=msid:oQrYP0RdXW3OITfGDlPmkOEuvWzFikm0VwlO a6613978-f98b-4fee-87b8-42f0ea143d1b
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:106 H264/90000
a=rtcp-fb:106 goog-remb
a=rtcp-fb:106 transport-cc
a=rtcp-fb:106 ccm fir
a=rtcp-fb:106 nack
a=rtcp-fb:106 nack pli
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=ssrc:2404631310 cname:g3pckgfqHMLdr/Ev
a=ssrc:2404631310 msid:oQrYP0RdXW3OITfGDlPmkOEuvWzFikm0VwlO a6613978-f98b-4fee-87b8-42f0ea143d1b
a=ssrc:2404631310 mslabel:oQrYP0RdXW3OITfGDlPmkOEuvWzFikm0VwlO
a=ssrc:2404631310 label:a6613978-f98b-4fee-87b8-42f0ea143d1b

...and the ANSWER from the Python app:

v=0
o=- 4514418055778671968 2 IN IP4 0.0.0.0
s=-
t=0 0
a=group:BUNDLE 0
m=video 9 UDP/TLS/RTP/SAVPF 106
c=IN IP4 0.0.0.0
a=ice-ufrag:dJWSQZTBciCBx2fEbfUoj+CfUl+MBBlF
a=ice-pwd:LI/7knZzqS/yUtOZDLU6P74mQDW/aCu3
a=mid:0
a=rtcp-mux
a=setup:active
a=rtpmap:106 H264/90000
a=rtcp-fb:106 nack pli
a=rtcp-fb:106 ccm fir
a=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=recvonly
a=fingerprint:sha-256 24:63:1E:23:6E:B0:A3:F3:1E:DA:C6:0E:7F:4F:DC:74:98:FE:F4:8D:AE:84:02:44:F1:16:F1:3D:C4:68:2B:25

That looks promising, because there seems to be an agreement on the format. What concerns me is the a=recvonly and in fact, the screen remains black on the Chrome side. And in case the Chrome app sends video as announced, the Python app crashes with sigsev after "on_incoming_pad_added".

I don't know if it means something: If I run the Python app as OFFERER this trace appears:

:00:03.684006943 16546 0xb284cf20 DEBUG              webrtcbin gstwebrtcbin.c:279:gst_webrtcbin_sink_event:<webrtcbin> On <webrtcbin:sink_0> checking negotiation? 1, caps application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, profile-level-id=(string)42c01f, sprop-parameter-sets=(string)"Z0LAH9kAUAW7AWoCAgKAAAADAIAAAB5HjBkk\,aMuMsg\=\=", payload=(int)96, seqnum-offset=(uint)20552, timestamp-offset=(uint)1267346039, ssrc=(uint)1634604066, a-framerate=(string)30

I see a difference in the profile-level-id of 42c01f compared to the later offered 42e01f, but it works, maybe because the Chrome decoder does not care. But on the Chrome side 42c01f is not in the range of possible codecs:

I'm a bit lost here and beginning to think about NOT using GST in ANSWERING mode, at least not for H.264... 

There seems to be no understanding between both parties...