webrtcbin doesn't retransmit because rtprtxsend has no payload type map

Wed Mar 16 00:49:06 UTC 2022

Hi,

On 16/3/22 03:09, Michiel Konstapel via gstreamer-devel wrote:
>
> Aha - it does appear to work if I set "bundle-policy" to "max-bundle".
> At least, then I see it setting up the pt map, and I see it
> creating/pushing rtx buffers. It doesn't seem to immediately work
> wonders for combating packet loss, though, but I do think it runs a
> little smoother.
>
> So RTX doesn't work with the default bundle-policy, is that correct?
>

Unknown.  If you're talking with a browser, then you can pretty much
always use max-bundle so the other modes don't get too much testing. 
The whole bundle-policy thing is also up for potential removal at some
point in the WebRTC specification.

> On 15-03-2022 16:50, Michiel Konstapel wrote:
>>
>> I see where webrtcbin should be setting the property, in
>> _set_rtx_ptmap_from_stream, but it's not, because both
>> stream->rtxreceive and stream->rtxsend are NULL at that point:
>>
>> webrtcbin
>> gstwebrtcbin.c:3707:_set_rtx_ptmap_from_stream:<transportstream0>
>> setting payload map on (NULL) : (NULL) and application/x-rtp-pt-map,
>> 96=(uint)97;
>>
>> Looks like this is the call from
>> gstwebrtcbin.c:on_rtpbin_request_aux_sender, but that only assigns
>> stream->rtxsend AFTER the call to _set_rtx_ptmap_from_stream.
>>
>> I'm confused - everything I read indicates that this should "just
>> work", but it doesn't for me, and I don't see how it even could.
>>

This was an issue that may have only been fixed in latest git.

>> On 15-03-2022 16:20, Michiel Konstapel wrote:
>>>
>>> Hi all,
>>>
>>> I am back at giving this another try: I am trying to get nack/rtx to
>>> work from webrtcbin to a browser, because even at 1% packet loss,
>>> video quality is terrible. I've continued from the situation below
>>> and did some more digging.
>>>
>>> The "not transmitted yet" message is incorrect: it is printing that
>>> because it is not finding the seqnum in the queue at all. The reason
>>> is, because nothing is ever getting put in! In process_buffer, it
>>> checks
>>>
>>> if (g_hash_table_contains (rtx->rtx_pt_map, GUINT_TO_POINTER
>>> (payload_type))) {
>>>
>>> but this is never true. I am never seeing the
>>> payload-type-mapproperty being set. According to the SDP, if I read
>>> it correctly, the retransmits for payload type 96 should be mapped
>>> to 97?
>>>
>>> m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0 c=IN IP4 0.0.0.0
>>> a=setup:actpass a=ice-ufrag:NlPtu+SaHS/sZZPE9QwSi5hAw+SAC3N/
>>> a=ice-pwd:75y3QklMjhg7UgWH6NcOKHkYT2s6hc5G a=rtcp-mux a=rtcp-rsize
>>> a=sendrecv a=rtpmap:96 H264/90000 a=rtcp-fb:96 nack a=rtcp-fb:96
>>> nack pli a=framerate:25 a=fmtp:96
>>> packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA==
>>> a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=ssrc-group:FID 1379148161
>>> 2265791741 a=ssrc:1379148161 msid:user1979923752 at host-8d116fd2
>>> webrtctransceiver0 a=ssrc:1379148161
>>> cname:user1979923752 at host-8d116fd2 a=ssrc:2265791741
>>> msid:user1979923752 at host-8d116fd2 webrtctransceiver0
>>> a=ssrc:2265791741 cname:user1979923752 at host-8d116fd2 a=mid:video0
>>>
>>> Chrome responds with the following answer:
>>>
>>> m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0
>>> c=IN IP4 0.0.0.0
>>> a=rtcp:9 IN IP4 0.0.0.0
>>> a=ice-ufrag:AaCC
>>> a=ice-pwd:JbPrnVgGCT7nyNDw2DSqjnMm
>>> a=ice-options:trickle
>>> a=fingerprint:sha-256
>>> C5:A8:41:97:10:C6:A7:C1:25:C1:62:51:CC:56:F6:81:9B:6F:1A:5A:3B:00:36:29:8E:9F:15:9F:E0:18:BC:DD
>>> a=setup:active
>>> a=mid:video0
>>> a=recvonly
>>> a=rtcp-mux
>>> a=rtcp-rsize
>>> a=rtpmap:96 H264/90000
>>> a=rtcp-fb:96 nack
>>> a=rtcp-fb:96 nack pli
>>> a=fmtp:96
>>> level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
>>> a=rtpmap:97 rtx/90000
>>> a=fmtp:97 apt=96
>>>
>>> All I am now doing to enable nack/rtx is to call
>>> transceiver.set_property("do-nack", True) on the video transceiver
>>> before calling create-offer. I could really use some advice on how
>>> to get this to work.
>>>
>>> Michiel
>>>
>>> On 04-10-2021 14:31, Michiel Konstapel wrote:
>>>> I am trying to improve WebRTC video quality in less than optimal
>>>> network conditions, sending audio and video from my pipeline to a
>>>> browser. My first thought was to try and enable do-nack on the
>>>> transceivers. Or for now, only the video transceiver, because
>>>> Chrome doesn't support NACK for audio?
>>>>
>>>> I am using tc to simulate 1% packet loss on the link, which has a
>>>> noticeable impact on the video (both visually (dropped frames and
>>>> smearing) and in terms of the packetsLost statistic).
>>>>
>>>> However, when I enable "do-nack", I see no improvement, and in the
>>>> logs I see:
>>>>
>>>> rtprtxsend
>>>> gstrtprtxsend.c:651:gst_rtp_rtx_send_sink_event:<rtprtxsend0>
>>>> Payload 96 not in rtx-pt-map
>>>> rtprtxsend
>>>> gstrtprtxsend.c:655:gst_rtp_rtx_send_sink_event:<rtprtxsend0> got
>>>> caps for payload: 96->-1, ssrc: 715396252->3436539345 :
>>>> application/x-rtp, media=(string)video, clock-rate=(int)90000,
>>>> encoding-name=(string)H264, packetization-mode=(string)1,
>>>> profile-level-id=(string)42c028,
>>>> sprop-parameter-sets=(string)"Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA\=\=\,aM48gA\=\=",
>>>> payload=(int)96, ssrc=(uint)715396252,
>>>> timestamp-offset=(uint)4223414689, seqnum-offset=(uint)25976,
>>>> a-framerate=(string)25
>>>>
>>>> on startup, and then
>>>>
>>>> rtprtxsend
>>>> gstrtprtxsend.c:524:gst_rtp_rtx_send_src_event:<rtprtxsend0>
>>>> requested seqnum 25100 has not been transmitted yet in the original
>>>> stream; either the remote end is not configured correctly, or the
>>>> source is too slow
>>>>
>>>> This repeats ~10 times for each sequence number.
>>>>
>>>> This is the generated offer:
>>>>
>>>> v=0
>>>> o=- 3673290479767803662 0 IN IP4 0.0.0.0
>>>> s=-
>>>> t=0 0
>>>> a=ice-options:trickle
>>>> m=video 9 UDP/TLS/RTP/SAVPF 96 97 98
>>>> c=IN IP4 0.0.0.0
>>>> a=setup:actpass
>>>> a=ice-ufrag:GBQSDSR0t2BjpCKzhBy6D5TWI5PIwHcL
>>>> a=ice-pwd:gVq+L+I5gtsYST5+0Ko1qiou+cmTMBek
>>>> a=rtcp-mux
>>>> a=rtcp-rsize
>>>> a=sendonly
>>>> a=rtpmap:96 H264/90000
>>>> a=rtcp-fb:96 nack pli
>>>> a=framerate:25
>>>> a=fmtp:96
>>>> packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA==
>>>> a=rtpmap:97 red/90000
>>>> a=rtpmap:98 ulpfec/90000
>>>> a=ssrc:694513671 msid:user119389484 at host-196b6568 webrtctransceiver2
>>>> a=ssrc:694513671 cname:user119389484 at host-196b6568
>>>> a=mid:video0
>>>> a=fingerprint:sha-256
>>>> 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F
>>>> m=audio 9 UDP/TLS/RTP/SAVPF 96
>>>> c=IN IP4 0.0.0.0
>>>> a=setup:actpass
>>>> a=ice-ufrag:DWJDEBPdaEDqrvwWH4taF+4IxwzJUbtt
>>>> a=ice-pwd:2TLZ/cUYHrYWec9NL2Q08z2X2N5pQEy6
>>>> a=rtcp-mux
>>>> a=rtcp-rsize
>>>> a=sendonly
>>>> a=rtpmap:96 OPUS/48000/2
>>>> a=rtcp-fb:96 nack pli
>>>> a=fmtp:96 sprop-maxcapturerate=48000;sprop-stereo=1
>>>> a=ssrc:1591833598 msid:user119389484 at host-196b6568 webrtctransceiver3
>>>> a=ssrc:1591833598 cname:user119389484 at host-196b6568
>>>> a=mid:audio1
>>>> a=fingerprint:sha-256
>>>> 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F
>>>>
>>>> This appears to also be offering forward error correction (red,
>>>> ulpfec)? However, I have not enabled that (yet) but I was going to
>>>> try that next.
>>>>
>>>> One thing that worries/confuses me, is that it is using payload
>>>> types 96/97/98 for video, and 96 for audio, too. Is the duplicate
>>>> 96 a problem? And, where is it getting these numbers? The RTP
>>>> packets I am using as "input" for webrtcbin have 96 for audio and
>>>> 97 for video.
>>>>
>>>> In short: how do I use NACKs or FEC to handle packet loss from
>>>> webrtcbin to a browser?
>>>>
>>>> Kind regards,
>>>> Michiel 
>>>
>> -- 
>> Michiel Konstapel
>> /Lead Software Developer/
>> aanmelder.nl <https://www.aanmelder.nl>
>>
>> T: +31(0)15 2400 119
>> E: michiel at aanmelder.nl
> -- 
> Michiel Konstapel
> /Lead Software Developer/
> aanmelder.nl <https://www.aanmelder.nl>
>
> T: +31(0)15 2400 119
> E: michiel at aanmelder.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220316/c29910c0/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20220316/c29910c0/attachment-0001.sig>