poor playback of MPEG-TS-over-UDP stream

[Tim Müller]

On Thu, 2016-02-04 at 14:16 -0700, Mark Howell wrote:

Hi Mark,

> I'm working with a network video widget that provides 
> H.264-on-MPEG-TS-on-UDP. I'm having a hard time getting reliable
> clean 
> decode and display of the video with low latency.
> 
> Using GStreamer 1.6.3 on Linux.
> 
> This pipeline works fine, plays back with reasonable latency and
> without 
> drops or glitches at 30 fps:
> 
> $ gst-launch-1.0 -v udpsrc port=15004 ! video/mpegts ! tsdemux ! 
> h264parse ! avdec_h264 ! fpsdisplaysink sync=false

Right, this (sync=false) basically disables timestamp and latency
handling at the sink, so everything is just processed and displayed as
fast as possible.

Which will probably be "ok" generally speaking, since the sender sends
video in real-time, but the video frames will not be timed correctly
according to their original timing, so there might be significant
amount of jitter. It might look smoothest, but it's probably far from
perfect.

drops or glitches most likely happen either because packets got lost
(or re-ordered) between sender and receiver, or because the default
size of the kernel-side udp buffer is rather small, and we're not
always reading the data out fast enough, in which case packets will be
dropped when the udp receive buffer overflows.

You can play with the buffer-size=N property of udpsrc, and you can
change the max allowed sizes for receive/send buffer like this (for
example):

  sudo /sbin/sysctl -w net.core.rmem_max=33554432
  sudo /sbin/sysctl -w net.core.wmem_max=33554432


> This doesn't work well, showing mangled frames occasionally, at 1-3
> fps:
> 
> $ gst-launch-1.0 -v udpsrc port=15004 ! video/mpegts ! tsdemux ! 
> h264parse ! avdec_h264 ! fpsdisplaysink sync=true

sync=true means timestamps and latency will be taken into account.
tsdemux will by default advertise a latency of 700ms, since that's the
worst case scenario allowed by the spec (in reality it will usually be
much smaller though). This means the sink will delay output by 700ms.
This means you need 700ms+ worth of buffering (plus decoder latency) in
the pipeline (including the UDP receive buffer) to make sure no data is
lost. Here you're relying solely on the UDP receive buffer to provide
that buffering. It's probably way too small, leading to lots of data
being dropped.


> I can make it better (but not perfect) by adding a queue, but that
> adds 
> 1 s or so of latency:
> 
> $ gst-launch-1.0 -v udpsrc port=15004 ! video/mpegts ! tsdemux ! 
> h264parse ! queue ! avdec_h264 ! fpsdisplaysink sync=true

A queue does not add latency. It adds a second worth of buffering, so
that should help prevent packets being dropped because they couldn't be
captured (but the receive buffer might still be so small that minor
scheduling differences can make enough of a difference that some data
will be dropped).

I would place the queue directly after udpsrc, or add another one
there.


> I suspect this is related to latency/timestamp handling in or near 
> tsdemux. I've seen commentary on bugzilla suggesting some 
> dissatisfaction with the state of that in tsdemux 
> (https://bugzilla.gnome.org/show_bug.cgi?id=608148) but don't
> understand it well enough atm to know what to try.
> 
> I made it behave *better* by changing the default latency from 700 ms
> to 50 ms in tsdemux.c (see attached patch). But again I don't
> understand it well enough to be clear *why*, or how to make it
> *right*.

If you know that's ok for your specific stream, that's probably fine.
The reason why it works better is hopefully clear from what I wrote
above.

> Plays OK (but not without debug complaints; perhaps they are
> relevant) 
> with vlc and libav:
> 
> $ cvlc udp://@:15004
> 
> and
> 
> $ avplay udp://*:15004
> 
> I'd appreciate some clues on how to proceed... I can run tests and
> grab  logs with some direction.

Might be an idea to capture the incoming data with wireshark or so as
well and dump it into a pcap file, then you can check if the incoming
data is problematic already or the issues are elsewhere.

Cheers
 -Tim

-- 
Tim Müller, Centricular Ltd - http://www.centricular.com