<div style="font-family: Arial; font-size: 14px;"></div><span>Dear all,</span><div><br></div><div><span>I have a quite complex streaming architecture where:</span></div><div><span>1. an analog infrared camera is connected to a custom board based on a Jetson TX2i</span></div><div><span>2. the board uses the Jetson's hw H.265 encoder and transmits the images via LTE as UDP stream</span></div><div><span>3. a VM receives the UDP stream, re-encodes it as H.264 and publishes a RTMP stream using NGINX (let's call it RTMP-VM)</span></div><div><span>4. clients connect to this RTMP stream and decode it locally.</span></div><div><br></div><div><span>The
purpose of the whole pipeline is to stream with the lowest latency and
with the lowest occupied LTE bandwidth possible (for the step (2) only,
(3) and (4) rely on high-speed connections) to multiple clients (which should be able to decode it easily --- that is, with a traditional player or, even better, youtube-style). I am therefore open to other alternatives, as long as they fit the two requirements above (and are supported by the gstreamer version on the board).</span></div><div><span>The gstreamer version on the
Jetson-based board is quite old (1.14 IIRC and, unfortunately --- due to
the atrocious support by NVIDIA --- no way to upgrade it), the other
machines are standard so I have a recent version.</span></div><div><br></div><div><span>I have tried streaming, and the latency is simply HUGE --- something in the order of 10 to 15 seconds !!</span></div><div><span>Network
latency, both for the uplink LTE link and the downlink wired link, is
in the order of 20-30 ms, and the pipelines (measured with
GST_DEBUG="GST_TRACER:7" GST_TRACERS=latency) take roughly 35 ms for the
encoding+UDP streaming on the Jetson, and between 10 and 30 ms for the
reencoding and RTMP retransmission, therefore I cannot explain myself
where this latency originates.<br></span></div><div><span>If, instead of
doing this re-encoding+RTMP streaming, I simply send the UDP packets to
the destination where they are decoded, the latency is in the order of
100-150 ms (which is acceptable for my application) --- but I have to
create a RTMP stream where "random" users can connect, thus this
(=multiudpsink) is not an option.<br></span></div><div><span>Also,
please note that I experience the same latency even when doing a test
with my PC streaming a low-res (256x144 pixels) video from my webcam to
the RTMP-VM, getting it back locally for viewing.</span></div><div><br></div><div><span>To
allow people reproducing my setup (and since I have no access to the
embedded board until next Tuesday), I give here the pipelines for the PC
streaming test (since the behaviour is exactly the same).</span></div><div><span>The
resolution of the stream from the embedded board is slightly higher
(720x288 pixels --- the occupied bw max out at roughly 1.3 Mbps) but
this does not seem to be the main issue. Also I limit the MTU in
rtph265pay since data passes through a VPN (and with this setting I
avoid packet fragmentation), but it does not seem to impact the latency
in any way.</span></div><br><div><span>IP of the RTMP-VM: 10.0.0.2</span></div><div><br></div><div><span>A. video encoding and upstream on my PC:</span></div><div><span>gst-launch-1.0
v4l2src ! jpegdec ! videoscale ! video/x-raw,width=256,height=144 !
x265enc ! rtph265pay mtu=970 ! udpsink host=10.0.0.2 port=5000</span></div><div><br></div><div><span>B. re-encoding and RTMP transmission on the RTMP VM:</span></div><div><span>gst-launch-1.0
udpsrc port=5000 ! queue min-threshold-time=20000000
max-size-time=100000000 ! "application/x-rtp, encoding-name=H265,
payload=96" ! rtph265depay ! h265parse ! avdec_h265 ! videoconvert !
x264enc sliced-threads=true tune=zerolatency ! h264parse ! flvmux
streamable=true ! queue max-size-time=0 ! rtmpsink
location='rtmp://127.0.0.1/live/stream live=true'</span></div><div><br></div><div><span>C. local decoding of the RTMP stream on my PC:</span></div><div><span>gst-launch-1.0 rtmpsrc location="rtmp://10.0.0.2/live/stream" ! decodebin ! autovideosink</span></div><div><br></div><div><span>If I use</span></div><div><span>gst-launch-1.0 uridecodebin uri=<span>"rtmp://10.0.0.2/live/stream"</span> buffer-duration=50000000 ! autovideosink</span></div><div><span>for (C), the pipeline gets paused all the time, printing on the terminal:</span></div><div><br></div><div><span>Buffering, setting pipeline to PAUSED ...</span></div><div><span>Done buffering, setting pipeline to PLAYING ...</span></div><div><br></div><div><span>(I don't know if this is normal, but I expect that if data was coming at full speed, the pipeline wouldn't have been paused...).</span></div><div><br></div><div><span>I
have literally spent days attempting every approach I have found on the
web and tweaking every single component of the pipelines, but to no
avail. Also, the fact that I cannot identify where this latency source
lies simply drives me nuts...</span></div><div><span>Can anyone help me, please?</span></div><div><span>Thanks a lot in advance !</span></div><div><br></div><div><span>Best,</span></div><div><span>Rob</span></div><div class="protonmail_signature_block" style="font-family: Arial; font-size: 14px;"><div class="protonmail_signature_block-proton"></div>
</div>