webrtcbin doesn't retransmit because rtprtxsend has no payload type map

Tue Mar 15 15:50:14 UTC 2022

I see where webrtcbin should be setting the property, in 
_set_rtx_ptmap_from_stream, but it's not, because both 
stream->rtxreceive and stream->rtxsend are NULL at that point:

webrtcbin 
gstwebrtcbin.c:3707:_set_rtx_ptmap_from_stream:<transportstream0> 
setting payload map on (NULL) : (NULL) and application/x-rtp-pt-map, 
96=(uint)97;

Looks like this is the call from 
gstwebrtcbin.c:on_rtpbin_request_aux_sender, but that only assigns 
stream->rtxsend AFTER the call to _set_rtx_ptmap_from_stream.

I'm confused - everything I read indicates that this should "just work", 
but it doesn't for me, and I don't see how it even could.

On 15-03-2022 16:20, Michiel Konstapel wrote:
>
> Hi all,
>
> I am back at giving this another try: I am trying to get nack/rtx to 
> work from webrtcbin to a browser, because even at 1% packet loss, 
> video quality is terrible. I've continued from the situation below and 
> did some more digging.
>
> The "not transmitted yet" message is incorrect: it is printing that 
> because it is not finding the seqnum in the queue at all. The reason 
> is, because nothing is ever getting put in! In process_buffer, it checks
>
> if (g_hash_table_contains (rtx->rtx_pt_map, GUINT_TO_POINTER 
> (payload_type))) {
>
> but this is never true. I am never seeing the payload-type-mapproperty 
> being set. According to the SDP, if I read it correctly, the 
> retransmits for payload type 96 should be mapped to 97?
>
> m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0 c=IN IP4 0.0.0.0 
> a=setup:actpass a=ice-ufrag:NlPtu+SaHS/sZZPE9QwSi5hAw+SAC3N/ 
> a=ice-pwd:75y3QklMjhg7UgWH6NcOKHkYT2s6hc5G a=rtcp-mux a=rtcp-rsize 
> a=sendrecv a=rtpmap:96 H264/90000 a=rtcp-fb:96 nack a=rtcp-fb:96 nack 
> pli a=framerate:25 a=fmtp:96 
> packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA== 
> a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=ssrc-group:FID 1379148161 
> 2265791741 a=ssrc:1379148161 msid:user1979923752 at host-8d116fd2 
> webrtctransceiver0 a=ssrc:1379148161 
> cname:user1979923752 at host-8d116fd2 a=ssrc:2265791741 
> msid:user1979923752 at host-8d116fd2 webrtctransceiver0 a=ssrc:2265791741 
> cname:user1979923752 at host-8d116fd2 a=mid:video0
>
> Chrome responds with the following answer:
>
> m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0
> c=IN IP4 0.0.0.0
> a=rtcp:9 IN IP4 0.0.0.0
> a=ice-ufrag:AaCC
> a=ice-pwd:JbPrnVgGCT7nyNDw2DSqjnMm
> a=ice-options:trickle
> a=fingerprint:sha-256 
> C5:A8:41:97:10:C6:A7:C1:25:C1:62:51:CC:56:F6:81:9B:6F:1A:5A:3B:00:36:29:8E:9F:15:9F:E0:18:BC:DD
> a=setup:active
> a=mid:video0
> a=recvonly
> a=rtcp-mux
> a=rtcp-rsize
> a=rtpmap:96 H264/90000
> a=rtcp-fb:96 nack
> a=rtcp-fb:96 nack pli
> a=fmtp:96 
> level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
> a=rtpmap:97 rtx/90000
> a=fmtp:97 apt=96
>
> All I am now doing to enable nack/rtx is to call 
> transceiver.set_property("do-nack", True) on the video transceiver 
> before calling create-offer. I could really use some advice on how to 
> get this to work.
>
> Michiel
>
> On 04-10-2021 14:31, Michiel Konstapel wrote:
>> I am trying to improve WebRTC video quality in less than optimal 
>> network conditions, sending audio and video from my pipeline to a 
>> browser. My first thought was to try and enable do-nack on the 
>> transceivers. Or for now, only the video transceiver, because Chrome 
>> doesn't support NACK for audio?
>>
>> I am using tc to simulate 1% packet loss on the link, which has a 
>> noticeable impact on the video (both visually (dropped frames and 
>> smearing) and in terms of the packetsLost statistic).
>>
>> However, when I enable "do-nack", I see no improvement, and in the 
>> logs I see:
>>
>> rtprtxsend 
>> gstrtprtxsend.c:651:gst_rtp_rtx_send_sink_event:<rtprtxsend0> Payload 
>> 96 not in rtx-pt-map
>> rtprtxsend 
>> gstrtprtxsend.c:655:gst_rtp_rtx_send_sink_event:<rtprtxsend0> got 
>> caps for payload: 96->-1, ssrc: 715396252->3436539345 : 
>> application/x-rtp, media=(string)video, clock-rate=(int)90000, 
>> encoding-name=(string)H264, packetization-mode=(string)1, 
>> profile-level-id=(string)42c028, 
>> sprop-parameter-sets=(string)"Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA\=\=\,aM48gA\=\=", 
>> payload=(int)96, ssrc=(uint)715396252, 
>> timestamp-offset=(uint)4223414689, seqnum-offset=(uint)25976, 
>> a-framerate=(string)25
>>
>> on startup, and then
>>
>> rtprtxsend 
>> gstrtprtxsend.c:524:gst_rtp_rtx_send_src_event:<rtprtxsend0> 
>> requested seqnum 25100 has not been transmitted yet in the original 
>> stream; either the remote end is not configured correctly, or the 
>> source is too slow
>>
>> This repeats ~10 times for each sequence number.
>>
>> This is the generated offer:
>>
>> v=0
>> o=- 3673290479767803662 0 IN IP4 0.0.0.0
>> s=-
>> t=0 0
>> a=ice-options:trickle
>> m=video 9 UDP/TLS/RTP/SAVPF 96 97 98
>> c=IN IP4 0.0.0.0
>> a=setup:actpass
>> a=ice-ufrag:GBQSDSR0t2BjpCKzhBy6D5TWI5PIwHcL
>> a=ice-pwd:gVq+L+I5gtsYST5+0Ko1qiou+cmTMBek
>> a=rtcp-mux
>> a=rtcp-rsize
>> a=sendonly
>> a=rtpmap:96 H264/90000
>> a=rtcp-fb:96 nack pli
>> a=framerate:25
>> a=fmtp:96 
>> packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA==
>> a=rtpmap:97 red/90000
>> a=rtpmap:98 ulpfec/90000
>> a=ssrc:694513671 msid:user119389484 at host-196b6568 webrtctransceiver2
>> a=ssrc:694513671 cname:user119389484 at host-196b6568
>> a=mid:video0
>> a=fingerprint:sha-256 
>> 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F
>> m=audio 9 UDP/TLS/RTP/SAVPF 96
>> c=IN IP4 0.0.0.0
>> a=setup:actpass
>> a=ice-ufrag:DWJDEBPdaEDqrvwWH4taF+4IxwzJUbtt
>> a=ice-pwd:2TLZ/cUYHrYWec9NL2Q08z2X2N5pQEy6
>> a=rtcp-mux
>> a=rtcp-rsize
>> a=sendonly
>> a=rtpmap:96 OPUS/48000/2
>> a=rtcp-fb:96 nack pli
>> a=fmtp:96 sprop-maxcapturerate=48000;sprop-stereo=1
>> a=ssrc:1591833598 msid:user119389484 at host-196b6568 webrtctransceiver3
>> a=ssrc:1591833598 cname:user119389484 at host-196b6568
>> a=mid:audio1
>> a=fingerprint:sha-256 
>> 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F
>>
>> This appears to also be offering forward error correction (red, 
>> ulpfec)? However, I have not enabled that (yet) but I was going to 
>> try that next.
>>
>> One thing that worries/confuses me, is that it is using payload types 
>> 96/97/98 for video, and 96 for audio, too. Is the duplicate 96 a 
>> problem? And, where is it getting these numbers? The RTP packets I am 
>> using as "input" for webrtcbin have 96 for audio and 97 for video.
>>
>> In short: how do I use NACKs or FEC to handle packet loss from 
>> webrtcbin to a browser?
>>
>> Kind regards,
>> Michiel 
>
-- 
Michiel Konstapel
/Lead Software Developer/
aanmelder.nl <https://www.aanmelder.nl>

T: +31(0)15 2400 119
E: michiel at aanmelder.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220315/2bfdd32b/attachment-0001.htm>