<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> I see where webrtcbin should be setting the property, in _set_rtx_ptmap_from_stream, but it's not, because both stream->rtxreceive and stream->rtxsend are NULL at that point: webrtcbin gstwebrtcbin.c:3707:_set_rtx_ptmap_from_stream:<transportstream0> setting payload map on (NULL) : (NULL) and application/x-rtp-pt-map, 96=(uint)97; Looks like this is the call from gstwebrtcbin.c:on_rtpbin_request_aux_sender, but that only assigns stream->rtxsend AFTER the call to _set_rtx_ptmap_from_stream. I'm confused - everything I read indicates that this should "just work", but it doesn't for me, and I don't see how it even could. <div class="moz-cite-prefix">On 15-03-2022 16:20, Michiel Konstapel wrote: </div> <blockquote type="cite" cite="mid:c011e2f8-1c4d-1dda-f287-6c4a1afd964f@aanmelder.nl"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> Hi all, I am back at giving this another try: I am trying to get nack/rtx to work from webrtcbin to a browser, because even at 1% packet loss, video quality is terrible. I've continued from the situation below and did some more digging. The "not transmitted yet" message is incorrect: it is printing that because it is not finding the seqnum in the queue at all. The reason is, because nothing is ever getting put in! In process_buffer, it checks if (g_hash_table_contains (rtx->rtx_pt_map, GUINT_TO_POINTER (payload_type))) { but this is never true. I am never seeing the payload-type-map property being set. According to the SDP, if I read it correctly, the retransmits for payload type 96 should be mapped to 97? <span style="color: rgb(0, 0, 0); font-family: monospace; font-size: 10.4px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0 c=IN IP4 0.0.0.0 a=setup:actpass a=ice-ufrag:NlPtu+SaHS/sZZPE9QwSi5hAw+SAC3N/ a=ice-pwd:75y3QklMjhg7UgWH6NcOKHkYT2s6hc5G a=rtcp-mux a=rtcp-rsize a=sendrecv a=rtpmap:96 H264/90000 a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=framerate:25 a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA== a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=ssrc-group:FID 1379148161 2265791741 a=ssrc:1379148161 msid:user1979923752@host-8d116fd2 webrtctransceiver0 a=ssrc:1379148161 cname:user1979923752@host-8d116fd2 a=ssrc:2265791741 msid:user1979923752@host-8d116fd2 webrtctransceiver0 a=ssrc:2265791741 cname:user1979923752@host-8d116fd2 a=<a class="moz-txt-link-freetext" href="mid:video0" moz-do-not-send="true">mid:video0</a> Chrome responds with the following answer: m=video 9 UDP/TLS/RTP/SAVPF 96 97 mid=video0 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:AaCC a=ice-pwd:JbPrnVgGCT7nyNDw2DSqjnMm a=ice-options:trickle a=fingerprint:sha-256 C5:A8:41:97:10:C6:A7:C1:25:C1:62:51:CC:56:F6:81:9B:6F:1A:5A:3B:00:36:29:8E:9F:15:9F:E0:18:BC:DD a=setup:active a=<a class="moz-txt-link-freetext" href="mid:video0" moz-do-not-send="true">mid:video0</a> a=recvonly a=rtcp-mux a=rtcp-rsize a=rtpmap:96 H264/90000 a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 All I am now doing to enable nack/rtx is to call transceiver.set_property("do-nack", True) on the video transceiver before calling create-offer. I could really use some advice on how to get this to work. Michiel <div class="moz-cite-prefix">On 04-10-2021 14:31, Michiel Konstapel wrote: </div> <blockquote type="cite" cite="mid:d841acc5-4d39-aaa1-ffdf-50fdb2a4c7b7@aanmelder.nl">I am trying to improve WebRTC video quality in less than optimal network conditions, sending audio and video from my pipeline to a browser. My first thought was to try and enable do-nack on the transceivers. Or for now, only the video transceiver, because Chrome doesn't support NACK for audio? I am using tc to simulate 1% packet loss on the link, which has a noticeable impact on the video (both visually (dropped frames and smearing) and in terms of the packetsLost statistic). However, when I enable "do-nack", I see no improvement, and in the logs I see: rtprtxsend gstrtprtxsend.c:651:gst_rtp_rtx_send_sink_event:<rtprtxsend0> Payload 96 not in rtx-pt-map rtprtxsend gstrtprtxsend.c:655:gst_rtp_rtx_send_sink_event:<rtprtxsend0> got caps for payload: 96->-1, ssrc: 715396252->3436539345 : application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, packetization-mode=(string)1, profile-level-id=(string)42c028, sprop-parameter-sets=(string)"Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA\=\=\,aM48gA\=\=", payload=(int)96, ssrc=(uint)715396252, timestamp-offset=(uint)4223414689, seqnum-offset=(uint)25976, a-framerate=(string)25 on startup, and then rtprtxsend gstrtprtxsend.c:524:gst_rtp_rtx_send_src_event:<rtprtxsend0> requested seqnum 25100 has not been transmitted yet in the original stream; either the remote end is not configured correctly, or the source is too slow This repeats ~10 times for each sequence number. This is the generated offer: v=0 o=- 3673290479767803662 0 IN IP4 0.0.0.0 s=- t=0 0 a=ice-options:trickle m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 c=IN IP4 0.0.0.0 a=setup:actpass a=ice-ufrag:GBQSDSR0t2BjpCKzhBy6D5TWI5PIwHcL a=ice-pwd:gVq+L+I5gtsYST5+0Ko1qiou+cmTMBek a=rtcp-mux a=rtcp-rsize a=sendonly a=rtpmap:96 H264/90000 a=rtcp-fb:96 nack pli a=framerate:25 a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKJWgHgCJ+XAWoCAgKAAAH0AABhqEIA==,aM48gA== a=rtpmap:97 red/90000 a=rtpmap:98 ulpfec/90000 a=ssrc:694513671 msid:user119389484@host-196b6568 webrtctransceiver2 a=ssrc:694513671 cname:user119389484@host-196b6568 a=<a class="moz-txt-link-freetext" href="mid:video0" moz-do-not-send="true">mid:video0</a> a=fingerprint:sha-256 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F m=audio 9 UDP/TLS/RTP/SAVPF 96 c=IN IP4 0.0.0.0 a=setup:actpass a=ice-ufrag:DWJDEBPdaEDqrvwWH4taF+4IxwzJUbtt a=ice-pwd:2TLZ/cUYHrYWec9NL2Q08z2X2N5pQEy6 a=rtcp-mux a=rtcp-rsize a=sendonly a=rtpmap:96 OPUS/48000/2 a=rtcp-fb:96 nack pli a=fmtp:96 sprop-maxcapturerate=48000;sprop-stereo=1 a=ssrc:1591833598 msid:user119389484@host-196b6568 webrtctransceiver3 a=ssrc:1591833598 cname:user119389484@host-196b6568 a=<a class="moz-txt-link-freetext" href="mid:audio1" moz-do-not-send="true">mid:audio1</a> a=fingerprint:sha-256 66:39:E9:A0:C1:C8:83:09:D0:3D:CD:A6:A7:1E:7B:A3:E4:DE:62:B0:20:08:98:16:74:F6:48:AD:49:BB:B9:0F This appears to also be offering forward error correction (red, ulpfec)? However, I have not enabled that (yet) but I was going to try that next. One thing that worries/confuses me, is that it is using payload types 96/97/98 for video, and 96 for audio, too. Is the duplicate 96 a problem? And, where is it getting these numbers? The RTP packets I am using as "input" for webrtcbin have 96 for audio and 97 for video. In short: how do I use NACKs or FEC to handle packet loss from webrtcbin to a browser? Kind regards, Michiel </blockquote> </blockquote> <div class="moz-signature">-- Michiel Konstapel Lead Software Developer <a href="https://www.aanmelder.nl">aanmelder.nl</a> T: +31(0)15 2400 119 E: <a class="moz-txt-link-abbreviated" href="mailto:michiel@aanmelder.nl">michiel@aanmelder.nl</a> </div> </body> </html>