<div>Dear Yu,</div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);"><br></div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);">Thanks for your reply, I wasn't aware of some of the options you've set and I'll gladly integrate them. <br></div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);">Actually, I am using the hw encoders on the board, the command I gave in my message was only to allow people to reproduce my problem without them.</div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);">In particular, on the board I use a C++ code I've written which does the following (+- some options of the different components, since I've tried tweaking everything I don't have a "standard" pipeline anymore.... but this should give you a rough idea):</div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);"><br></div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);"><span>gst-launch-1.0 v4l2src device="/dev/video0" ! video/x-raw,width=720,height=288,format=YUY2 ! videoconvert ! "video/x-raw, format=GRAY8" ! nvvidconv ! "video/x-raw(memory:NVMM), format=I420" ! nvv4l2h265enc ! rtph265pay ! udpsink port=5000 host=10.0.0.2</span></div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);"><br></div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);">When I use point-to-point communication with UDP (that is, I encode on the board and decode+display on the UDP sink side), latency is just fine (roughly 100-150 ms, depending on network delays, which is fine for my application). However, when I add the intermediate step that does the RTMP retransmission, latency skyrockets to 10-15 seconds :(</div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);"><br></div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);">Any clue about where this buffering is occurring?</div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);"><br></div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);">Thanks a lot in advance and have a nice day!</div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);"><br></div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);">Best,</div><div style="font-family: Arial; font-size: 14px; color: rgb(0, 0, 0);">Rob<br></div><div class="protonmail_quote">
------- Original Message -------<br>
On Thursday, August 18th, 2022 at 4:37 PM, Yu You <youyu.youyu@gmail.com> wrote:<br><br>
<blockquote class="protonmail_quote" type="cite">
<div dir="ltr">Usually, the default attributes of the x265enc (and H264 encoder) are not ideally suitable for real-time use cases. Try something, e.g. fixed GOP (key-int-max=30), speed-preset=superfast tune=zerolatency etc. In addition, <br>* rtph265pay aggregate-mode=aggregate-mode;<br><div>* use stream-format=byte-stream;<br><div>* use "h265parse config-interval=-1";<br>* add queue before and after encoder;<div><br></div><div>Alternatively, try to use HW encoders like NVCodec provided if using NVidia cards. </div><div><br></div><div>Good luck,</div><div><br></div><div>Yu </div></div></div></div><br><div class="gmail_quote"><div class="gmail_attr" dir="ltr">On Thu, 18 Aug 2022 at 15:33, robxkx7 via gstreamer-devel <<a href="mailto:gstreamer-devel@lists.freedesktop.org" rel="noreferrer nofollow noopener" target="_blank">gstreamer-devel@lists.freedesktop.org</a>> wrote:<br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div style="font-family:Arial;font-size:14px"></div><span>Dear all,</span><div><br></div><div><span>I have a quite complex streaming architecture where:</span></div><div><span>1. an analog infrared camera is connected to a custom board based on a Jetson TX2i</span></div><div><span>2. the board uses the Jetson's hw H.265 encoder and transmits the images via LTE as UDP stream</span></div><div><span>3. a VM receives the UDP stream, re-encodes it as H.264 and publishes a RTMP stream using NGINX (let's call it RTMP-VM)</span></div><div><span>4. clients connect to this RTMP stream and decode it locally.</span></div><div><br></div><div><span>The
purpose of the whole pipeline is to stream with the lowest latency and
with the lowest occupied LTE bandwidth possible (for the step (2) only,
(3) and (4) rely on high-speed connections) to multiple clients (which should be able to decode it easily --- that is, with a traditional player or, even better, youtube-style). I am therefore open to other alternatives, as long as they fit the two requirements above (and are supported by the gstreamer version on the board).</span></div><div><span>The gstreamer version on the
Jetson-based board is quite old (1.14 IIRC and, unfortunately --- due to
the atrocious support by NVIDIA --- no way to upgrade it), the other
machines are standard so I have a recent version.</span></div><div><br></div><div><span>I have tried streaming, and the latency is simply HUGE --- something in the order of 10 to 15 seconds !!</span></div><div><span>Network
latency, both for the uplink LTE link and the downlink wired link, is
in the order of 20-30 ms, and the pipelines (measured with
GST_DEBUG="GST_TRACER:7" GST_TRACERS=latency) take roughly 35 ms for the
encoding+UDP streaming on the Jetson, and between 10 and 30 ms for the
reencoding and RTMP retransmission, therefore I cannot explain myself
where this latency originates.<br></span></div><div><span>If, instead of
doing this re-encoding+RTMP streaming, I simply send the UDP packets to
the destination where they are decoded, the latency is in the order of
100-150 ms (which is acceptable for my application) --- but I have to
create a RTMP stream where "random" users can connect, thus this
(=multiudpsink) is not an option.<br></span></div><div><span>Also,
please note that I experience the same latency even when doing a test
with my PC streaming a low-res (256x144 pixels) video from my webcam to
the RTMP-VM, getting it back locally for viewing.</span></div><div><br></div><div><span>To
allow people reproducing my setup (and since I have no access to the
embedded board until next Tuesday), I give here the pipelines for the PC
streaming test (since the behaviour is exactly the same).</span></div><div><span>The
resolution of the stream from the embedded board is slightly higher
(720x288 pixels --- the occupied bw max out at roughly 1.3 Mbps) but
this does not seem to be the main issue. Also I limit the MTU in
rtph265pay since data passes through a VPN (and with this setting I
avoid packet fragmentation), but it does not seem to impact the latency
in any way.</span></div><br><div><span>IP of the RTMP-VM: 10.0.0.2</span></div><div><br></div><div><span>A. video encoding and upstream on my PC:</span></div><div><span>gst-launch-1.0
v4l2src ! jpegdec ! videoscale ! video/x-raw,width=256,height=144 !
x265enc ! rtph265pay mtu=970 ! udpsink host=10.0.0.2 port=5000</span></div><div><br></div><div><span>B. re-encoding and RTMP transmission on the RTMP VM:</span></div><div><span>gst-launch-1.0
udpsrc port=5000 ! queue min-threshold-time=20000000
max-size-time=100000000 ! "application/x-rtp, encoding-name=H265,
payload=96" ! rtph265depay ! h265parse ! avdec_h265 ! videoconvert !
x264enc sliced-threads=true tune=zerolatency ! h264parse ! flvmux
streamable=true ! queue max-size-time=0 ! rtmpsink
location='rtmp://<a target="_blank" href="http://127.0.0.1/live/stream" rel="noreferrer nofollow noopener">127.0.0.1/live/stream</a> live=true'</span></div><div><br></div><div><span>C. local decoding of the RTMP stream on my PC:</span></div><div><span>gst-launch-1.0 rtmpsrc location="rtmp://<a target="_blank" href="http://10.0.0.2/live/stream" rel="noreferrer nofollow noopener">10.0.0.2/live/stream</a>" ! decodebin ! autovideosink</span></div><div><br></div><div><span>If I use</span></div><div><span>gst-launch-1.0 uridecodebin uri=<span>"rtmp://<a target="_blank" href="http://10.0.0.2/live/stream" rel="noreferrer nofollow noopener">10.0.0.2/live/stream</a>"</span> buffer-duration=50000000 ! autovideosink</span></div><div><span>for (C), the pipeline gets paused all the time, printing on the terminal:</span></div><div><br></div><div><span>Buffering, setting pipeline to PAUSED ...</span></div><div><span>Done buffering, setting pipeline to PLAYING ...</span></div><div><br></div><div><span>(I don't know if this is normal, but I expect that if data was coming at full speed, the pipeline wouldn't have been paused...).</span></div><div><br></div><div><span>I
have literally spent days attempting every approach I have found on the
web and tweaking every single component of the pipelines, but to no
avail. Also, the fact that I cannot identify where this latency source
lies simply drives me nuts...</span></div><div><span>Can anyone help me, please?</span></div><div><span>Thanks a lot in advance !</span></div><div><br></div><div><span>Best,</span></div><div><span>Rob</span></div><div style="font-family:Arial;font-size:14px"><div></div>
</div>
</blockquote></div>
</blockquote><br>
</div>