<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hey,<br>
<br>
<div class="moz-cite-prefix">On 05/18/2018 07:59 AM, Lauri
Ehrenpreis wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CABvtbUVoD=+TmpmTbOdG_v7xUiRdk6qjyVZqCgmwhxTpNXG=4A@mail.gmail.com">
<div dir="ltr">ULP does not always protect only part of the
packets. It can also protect full packets. In fact the current
implementation of gstrtpulpfecenc.c is only protecting full
packets. </div>
</blockquote>
<br>
Yes, I am aware of that, but the decoder isn't necessarily :)<br>
<br>
<blockquote type="cite"
cite="mid:CABvtbUVoD=+TmpmTbOdG_v7xUiRdk6qjyVZqCgmwhxTpNXG=4A@mail.gmail.com">
<div dir="ltr">Also the fec decored knows if it recovered full
packet or only part of the pecket. <br>
</div>
</blockquote>
<br>
That's interesting, for context I was the one that upstreamed the
ulpfec elements,<br>
but I didn't do the implementation, how can the fec decoder
determine that?<br>
<br>
<blockquote type="cite"
cite="mid:CABvtbUVoD=+TmpmTbOdG_v7xUiRdk6qjyVZqCgmwhxTpNXG=4A@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>Given this I see no reason why FEC decoder should be placed
downstream from jitterbuffer. Being downstream and only
working when jitterbuffer reported loss means that we are not
getting any latency benefit from FEC as it only recovers
packet _after_ jitterbuffer timed out the packet and reported
loss. Jitterbuffer latency can be very large - 2 sec for
example. This means that even if we paid for the bandwidth and
sent FEC, we still get 2 sec delay. Also additional bandwidth
will be wasted because <span
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">RTX</span>
will anyway retransmit the packet which was already recovered
by FEC. So double loss because of current implementation.</div>
</div>
</blockquote>
<br>
I assume that this concern only exists when using
faststart-min-packets, correct?<br>
<br>
<blockquote type="cite"
cite="mid:CABvtbUVoD=+TmpmTbOdG_v7xUiRdk6qjyVZqCgmwhxTpNXG=4A@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>To make RTX and FEC work together I placed FEC decoder
upstream from jitterbuffer (using request-aux-receiver which
returns a bin containing rtprtxreceive, rtpstorage and
rtpulpfecdec). Also I modified FEC decoder so that it looks
for the sequence number gaps in storage on every incoming
packet and tries to recover those immediately. This way FEC
will reduce the latency if it recovers something and also it
can work together with rtx. There are minor optimizations to
be done since for example currently jitterbuffer also requests
retransmit for FEC packets. These requests should be ignored
on sender side. </div>
<div><br>
</div>
<div>I think in case it's not known if FEC will be able to
recover full or only partial packets, there should be a FEC
decoder before jitterbuffer which only recovers full packets
and another one downstream which recovers partial packets when
jitterbuffer considers those lost.</div>
</div>
</blockquote>
<br>
That could be a nice approach, as far as I understand all the
machinery is already<br>
in place in rtpbin to support this, the only thing that would be
needed is a special<br>
mode in ulpfecdec, detecting seqnum gaps and trying to do preemptive
recovery<br>
of packets it knows it can fully recover (this hinges on my earlier
question).<br>
<br>
Alternatively, if we can indeed determine whether the recovered
packets are fully<br>
recoverable, we can probably dispense with the downstream decoder
altogether,<br>
by tagging the recovered buffers in a way that the jitter buffer can
interpret to<br>
decide whether it should send retransmission requests nevertheless,
replacing<br>
the recovered packet with the original if it arrives late or is
retransmitted.<br>
<br>
<div class="moz-signature">-- <br>
Mathieu Duponchelle · <a href="https://www.centricular.com">https://www.centricular.com</a></div>
<blockquote type="cite"
cite="mid:CABvtbUVoD=+TmpmTbOdG_v7xUiRdk6qjyVZqCgmwhxTpNXG=4A@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>--</div>
<div>LauriE</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, May 16, 2018 at 3:43 PM,
Mathieu Duponchelle <span dir="ltr"><<a
href="mailto:mathieu@centricular.com" target="_blank"
moz-do-not-send="true">mathieu@centricular.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Lauri,<br>
<span class=""><br>
<br>
<br>
On 05/14/2018 02:53 PM, Lauri Ehrenpreis wrote:<br>
> Hi!<br>
><br>
> I tried to set up a pipeline which does both
retransmissions and FEC for video. My hope was that first
pipeline will try to recover packet with FEC and if that
fails it will request retransmission. As there was no
working example for such setup I
used gst-plugins-good/tests/<wbr>examples/rtp/client-rtpaux.c
and gst-plugins-good/tests/<wbr>examples/rtp/server-rtpaux.c
as basis and added request-fec-decoder
& request-fec-encoder signal handlers there. <br>
><br>
> After adding FEC handlers gstrtpbin came up with
following pipeline on client side (attached image). So
retransmission component is before jitterbuffer and FEC is
after. Do I understand correctly the pipeline will first
try to request retransmission if packet is lost and only
after the retransmission did not deliver the packet in
time for jitterbuffer, it will try use FEC to recover
packet? Why it is done this way - wouldn't it be better to
try FEC first?<br>
<br>
</span>Yes, you are correct, FEC recovery will only be
attempted once a packet is actually<br>
considered lost.<br>
<br>
While we could indeed reconstruct packets as soon as they
are late, there is no<br>
guarantee that the reconstructed packet would be "complete",
at least with ULPFEC.<br>
<br>
As you might already know, ULP stands for "uneven level
protection", and works based<br>
on the assumption that with most video codecs / payloaders,
the most important part<br>
of a packet is at the start, which means that a percentage
of protection can be applied<br>
at the packet level by only protecting the initial portion,
possibly making exceptions<br>
for keyframe packets for example, which could be fully
protected.<br>
<br>
This means that while we could reconstruct a packet, if
retransmission has been<br>
enabled it makes sense to ask for that packet to be
retransmitted anyway.<br>
<br>
I hope that made sense :)<br>
<span class="HOEnZb"><font color="#888888"><br>
-- <br>
Mathieu Duponchelle · <a
href="https://www.centricular.com" rel="noreferrer"
target="_blank" moz-do-not-send="true">https://www.centricular.com</a><br>
</font></span></blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
gstreamer-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a>
<a class="moz-txt-link-freetext" href="https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a>
</pre>
</blockquote>
</body>
</html>