webrtcbin doesn't retransmit because rtprtxsend has no payload type map

Tue Mar 15 16:09:10 UTC 2022

Aha - it does appear to work if I set "bundle-policy" to "max-bundle". 
At least, then I see it setting up the pt map, and I see it 
creating/pushing rtx buffers. It doesn't seem to immediately work 
wonders for combating packet loss, though, but I do think it runs a 
little smoother.

So RTX doesn't work with the default bundle-policy, is that correct?

On 15-03-2022 16:50, Michiel Konstapel wrote:
>
> I see where webrtcbin should be setting the property, in 
> _set_rtx_ptmap_from_stream, but it's not, because both 
> stream->rtxreceive and stream->rtxsend are NULL at that point:
>
> webrtcbin 
> gstwebrtcbin.c:3707:_set_rtx_ptmap_from_stream:<transportstream0> 
> setting payload map on (NULL) : (NULL) and application/x-rtp-pt-map, 
> 96=(uint)97;
>
> Looks like this is the call from 
> gstwebrtcbin.c:on_rtpbin_request_aux_sender, but that only assigns 
> stream->rtxsend AFTER the call to _set_rtx_ptmap_from_stream.
>
> I'm confused - everything I read indicates that this should "just 
> work", but it doesn't for me, and I don't see how it even could.
>
> On 15-03-2022 16:20, Michiel Konstapel wrote:
>>
>> Hi all,
>>
>> I am back at giving this another try: I am trying to get nack/rtx to 
>> work from webrtcbin to a browser, because even at 1% packet loss, 
>> video quality is terrible. I've continued from the situation below 
>> and did some more digging.
>>
>> The "not transmitted yet" message is incorrect: it is printing that 
>> because it is not finding the seqnum in the queue at all. The reason 
>> is, because nothing is ever getting put in! In process_buffer, it checks
>>
>> if (g_hash_table_contains (rtx->rtx_pt_map, GUINT_TO_POINTER 
>> (payload_type))) {
>>
>> but this is never true. I am never seeing the 
>> payload-type-mapproperty being set. According to the SDP, if I read 
>> it correctly, the retransmits for payload type 96 should be mapped to 97?
>>
>> m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0 c=IN IP4 0.0.0.0 
>> a=setup:actpass a=ice-ufrag:NlPtu+SaHS/sZZPE9QwSi5hAw+SAC3N/ 
>> a=ice-pwd:75y3QklMjhg7UgWH6NcOKHkYT2s6hc5G a=rtcp-mux a=rtcp-rsize 
>> a=sendrecv a=rtpmap:96 H264/90000 a=rtcp-fb:96 nack a=rtcp-fb:96 nack 
>> pli a=framerate:25 a=fmtp:96 
>> packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA== 
>> a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=ssrc-group:FID 1379148161 
>> 2265791741 a=ssrc:1379148161 msid:user1979923752 at host-8d116fd2 
>> webrtctransceiver0 a=ssrc:1379148161 
>> cname:user1979923752 at host-8d116fd2 a=ssrc:2265791741 
>> msid:user1979923752 at host-8d116fd2 webrtctransceiver0 
>> a=ssrc:2265791741 cname:user1979923752 at host-8d116fd2 a=mid:video0
>>
>> Chrome responds with the following answer:
>>
>> m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0
>> c=IN IP4 0.0.0.0
>> a=rtcp:9 IN IP4 0.0.0.0
>> a=ice-ufrag:AaCC
>> a=ice-pwd:JbPrnVgGCT7nyNDw2DSqjnMm
>> a=ice-options:trickle
>> a=fingerprint:sha-256 
>> C5:A8:41:97:10:C6:A7:C1:25:C1:62:51:CC:56:F6:81:9B:6F:1A:5A:3B:00:36:29:8E:9F:15:9F:E0:18:BC:DD
>> a=setup:active
>> a=mid:video0
>> a=recvonly
>> a=rtcp-mux
>> a=rtcp-rsize
>> a=rtpmap:96 H264/90000
>> a=rtcp-fb:96 nack
>> a=rtcp-fb:96 nack pli
>> a=fmtp:96 
>> level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
>> a=rtpmap:97 rtx/90000
>> a=fmtp:97 apt=96
>>
>> All I am now doing to enable nack/rtx is to call 
>> transceiver.set_property("do-nack", True) on the video transceiver 
>> before calling create-offer. I could really use some advice on how to 
>> get this to work.
>>
>> Michiel
>>
>> On 04-10-2021 14:31, Michiel Konstapel wrote:
>>> I am trying to improve WebRTC video quality in less than optimal 
>>> network conditions, sending audio and video from my pipeline to a 
>>> browser. My first thought was to try and enable do-nack on the 
>>> transceivers. Or for now, only the video transceiver, because Chrome 
>>> doesn't support NACK for audio?
>>>
>>> I am using tc to simulate 1% packet loss on the link, which has a 
>>> noticeable impact on the video (both visually (dropped frames and 
>>> smearing) and in terms of the packetsLost statistic).
>>>
>>> However, when I enable "do-nack", I see no improvement, and in the 
>>> logs I see:
>>>
>>> rtprtxsend 
>>> gstrtprtxsend.c:651:gst_rtp_rtx_send_sink_event:<rtprtxsend0> 
>>> Payload 96 not in rtx-pt-map
>>> rtprtxsend 
>>> gstrtprtxsend.c:655:gst_rtp_rtx_send_sink_event:<rtprtxsend0> got 
>>> caps for payload: 96->-1, ssrc: 715396252->3436539345 : 
>>> application/x-rtp, media=(string)video, clock-rate=(int)90000, 
>>> encoding-name=(string)H264, packetization-mode=(string)1, 
>>> profile-level-id=(string)42c028, 
>>> sprop-parameter-sets=(string)"Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA\=\=\,aM48gA\=\=", 
>>> payload=(int)96, ssrc=(uint)715396252, 
>>> timestamp-offset=(uint)4223414689, seqnum-offset=(uint)25976, 
>>> a-framerate=(string)25
>>>
>>> on startup, and then
>>>
>>> rtprtxsend 
>>> gstrtprtxsend.c:524:gst_rtp_rtx_send_src_event:<rtprtxsend0> 
>>> requested seqnum 25100 has not been transmitted yet in the original 
>>> stream; either the remote end is not configured correctly, or the 
>>> source is too slow
>>>
>>> This repeats ~10 times for each sequence number.
>>>
>>> This is the generated offer:
>>>
>>> v=0
>>> o=- 3673290479767803662 0 IN IP4 0.0.0.0
>>> s=-
>>> t=0 0
>>> a=ice-options:trickle
>>> m=video 9 UDP/TLS/RTP/SAVPF 96 97 98
>>> c=IN IP4 0.0.0.0
>>> a=setup:actpass
>>> a=ice-ufrag:GBQSDSR0t2BjpCKzhBy6D5TWI5PIwHcL
>>> a=ice-pwd:gVq+L+I5gtsYST5+0Ko1qiou+cmTMBek
>>> a=rtcp-mux
>>> a=rtcp-rsize
>>> a=sendonly
>>> a=rtpmap:96 H264/90000
>>> a=rtcp-fb:96 nack pli
>>> a=framerate:25
>>> a=fmtp:96 
>>> packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA==
>>> a=rtpmap:97 red/90000
>>> a=rtpmap:98 ulpfec/90000
>>> a=ssrc:694513671 msid:user119389484 at host-196b6568 webrtctransceiver2
>>> a=ssrc:694513671 cname:user119389484 at host-196b6568
>>> a=mid:video0
>>> a=fingerprint:sha-256 
>>> 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F
>>> m=audio 9 UDP/TLS/RTP/SAVPF 96
>>> c=IN IP4 0.0.0.0
>>> a=setup:actpass
>>> a=ice-ufrag:DWJDEBPdaEDqrvwWH4taF+4IxwzJUbtt
>>> a=ice-pwd:2TLZ/cUYHrYWec9NL2Q08z2X2N5pQEy6
>>> a=rtcp-mux
>>> a=rtcp-rsize
>>> a=sendonly
>>> a=rtpmap:96 OPUS/48000/2
>>> a=rtcp-fb:96 nack pli
>>> a=fmtp:96 sprop-maxcapturerate=48000;sprop-stereo=1
>>> a=ssrc:1591833598 msid:user119389484 at host-196b6568 webrtctransceiver3
>>> a=ssrc:1591833598 cname:user119389484 at host-196b6568
>>> a=mid:audio1
>>> a=fingerprint:sha-256 
>>> 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F
>>>
>>> This appears to also be offering forward error correction (red, 
>>> ulpfec)? However, I have not enabled that (yet) but I was going to 
>>> try that next.
>>>
>>> One thing that worries/confuses me, is that it is using payload 
>>> types 96/97/98 for video, and 96 for audio, too. Is the duplicate 96 
>>> a problem? And, where is it getting these numbers? The RTP packets I 
>>> am using as "input" for webrtcbin have 96 for audio and 97 for video.
>>>
>>> In short: how do I use NACKs or FEC to handle packet loss from 
>>> webrtcbin to a browser?
>>>
>>> Kind regards,
>>> Michiel 
>>
> -- 
> Michiel Konstapel
> /Lead Software Developer/
> aanmelder.nl <https://www.aanmelder.nl>
>
> T: +31(0)15 2400 119
> E: michiel at aanmelder.nl
-- 
Michiel Konstapel
/Lead Software Developer/
aanmelder.nl <https://www.aanmelder.nl>

T: +31(0)15 2400 119
E: michiel at aanmelder.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220315/8bb04832/attachment.htm>