webrtcbin doesn't retransmit because rtprtxsend has no payload type map

Tue Mar 15 15:20:41 UTC 2022

Hi all,

I am back at giving this another try: I am trying to get nack/rtx to 
work from webrtcbin to a browser, because even at 1% packet loss, video 
quality is terrible. I've continued from the situation below and did 
some more digging.

The "not transmitted yet" message is incorrect: it is printing that 
because it is not finding the seqnum in the queue at all. The reason is, 
because nothing is ever getting put in! In process_buffer, it checks

if (g_hash_table_contains (rtx->rtx_pt_map, GUINT_TO_POINTER 
(payload_type))) {

but this is never true. I am never seeing the payload-type-mapproperty 
being set. According to the SDP, if I read it correctly, the retransmits 
for payload type 96 should be mapped to 97?

m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0 c=IN IP4 0.0.0.0 
a=setup:actpass a=ice-ufrag:NlPtu+SaHS/sZZPE9QwSi5hAw+SAC3N/ 
a=ice-pwd:75y3QklMjhg7UgWH6NcOKHkYT2s6hc5G a=rtcp-mux a=rtcp-rsize 
a=sendrecv a=rtpmap:96 H264/90000 a=rtcp-fb:96 nack a=rtcp-fb:96 nack 
pli a=framerate:25 a=fmtp:96 
packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA== 
a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=ssrc-group:FID 1379148161 
2265791741 a=ssrc:1379148161 msid:user1979923752 at host-8d116fd2 
webrtctransceiver0 a=ssrc:1379148161 cname:user1979923752 at host-8d116fd2 
a=ssrc:2265791741 msid:user1979923752 at host-8d116fd2 webrtctransceiver0 
a=ssrc:2265791741 cname:user1979923752 at host-8d116fd2 a=mid:video0

Chrome responds with the following answer:

m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:AaCC
a=ice-pwd:JbPrnVgGCT7nyNDw2DSqjnMm
a=ice-options:trickle
a=fingerprint:sha-256 
C5:A8:41:97:10:C6:A7:C1:25:C1:62:51:CC:56:F6:81:9B:6F:1A:5A:3B:00:36:29:8E:9F:15:9F:E0:18:BC:DD
a=setup:active
a=mid:video0
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=fmtp:96 
level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96

All I am now doing to enable nack/rtx is to call 
transceiver.set_property("do-nack", True) on the video transceiver 
before calling create-offer. I could really use some advice on how to 
get this to work.

Michiel

On 04-10-2021 14:31, Michiel Konstapel wrote:
> I am trying to improve WebRTC video quality in less than optimal 
> network conditions, sending audio and video from my pipeline to a 
> browser. My first thought was to try and enable do-nack on the 
> transceivers. Or for now, only the video transceiver, because Chrome 
> doesn't support NACK for audio?
>
> I am using tc to simulate 1% packet loss on the link, which has a 
> noticeable impact on the video (both visually (dropped frames and 
> smearing) and in terms of the packetsLost statistic).
>
> However, when I enable "do-nack", I see no improvement, and in the 
> logs I see:
>
> rtprtxsend 
> gstrtprtxsend.c:651:gst_rtp_rtx_send_sink_event:<rtprtxsend0> Payload 
> 96 not in rtx-pt-map
> rtprtxsend 
> gstrtprtxsend.c:655:gst_rtp_rtx_send_sink_event:<rtprtxsend0> got caps 
> for payload: 96->-1, ssrc: 715396252->3436539345 : application/x-rtp, 
> media=(string)video, clock-rate=(int)90000, 
> encoding-name=(string)H264, packetization-mode=(string)1, 
> profile-level-id=(string)42c028, 
> sprop-parameter-sets=(string)"Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA\=\=\,aM48gA\=\=", 
> payload=(int)96, ssrc=(uint)715396252, 
> timestamp-offset=(uint)4223414689, seqnum-offset=(uint)25976, 
> a-framerate=(string)25
>
> on startup, and then
>
> rtprtxsend 
> gstrtprtxsend.c:524:gst_rtp_rtx_send_src_event:<rtprtxsend0> requested 
> seqnum 25100 has not been transmitted yet in the original stream; 
> either the remote end is not configured correctly, or the source is 
> too slow
>
> This repeats ~10 times for each sequence number.
>
> This is the generated offer:
>
> v=0
> o=- 3673290479767803662 0 IN IP4 0.0.0.0
> s=-
> t=0 0
> a=ice-options:trickle
> m=video 9 UDP/TLS/RTP/SAVPF 96 97 98
> c=IN IP4 0.0.0.0
> a=setup:actpass
> a=ice-ufrag:GBQSDSR0t2BjpCKzhBy6D5TWI5PIwHcL
> a=ice-pwd:gVq+L+I5gtsYST5+0Ko1qiou+cmTMBek
> a=rtcp-mux
> a=rtcp-rsize
> a=sendonly
> a=rtpmap:96 H264/90000
> a=rtcp-fb:96 nack pli
> a=framerate:25
> a=fmtp:96 
> packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA==
> a=rtpmap:97 red/90000
> a=rtpmap:98 ulpfec/90000
> a=ssrc:694513671 msid:user119389484 at host-196b6568 webrtctransceiver2
> a=ssrc:694513671 cname:user119389484 at host-196b6568
> a=mid:video0
> a=fingerprint:sha-256 
> 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F
> m=audio 9 UDP/TLS/RTP/SAVPF 96
> c=IN IP4 0.0.0.0
> a=setup:actpass
> a=ice-ufrag:DWJDEBPdaEDqrvwWH4taF+4IxwzJUbtt
> a=ice-pwd:2TLZ/cUYHrYWec9NL2Q08z2X2N5pQEy6
> a=rtcp-mux
> a=rtcp-rsize
> a=sendonly
> a=rtpmap:96 OPUS/48000/2
> a=rtcp-fb:96 nack pli
> a=fmtp:96 sprop-maxcapturerate=48000;sprop-stereo=1
> a=ssrc:1591833598 msid:user119389484 at host-196b6568 webrtctransceiver3
> a=ssrc:1591833598 cname:user119389484 at host-196b6568
> a=mid:audio1
> a=fingerprint:sha-256 
> 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F
>
> This appears to also be offering forward error correction (red, 
> ulpfec)? However, I have not enabled that (yet) but I was going to try 
> that next.
>
> One thing that worries/confuses me, is that it is using payload types 
> 96/97/98 for video, and 96 for audio, too. Is the duplicate 96 a 
> problem? And, where is it getting these numbers? The RTP packets I am 
> using as "input" for webrtcbin have 96 for audio and 97 for video.
>
> In short: how do I use NACKs or FEC to handle packet loss from 
> webrtcbin to a browser?
>
> Kind regards,
> Michiel 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220315/1bc56e56/attachment.htm>