kmssink and ultra low latency

[Peter Allen]

I have an application that needs ultra-low latency decoding and display of
a 720p/60 H264 rtp stream. I'm using Allwinner H3 (currently an Orange Pi
PC, but that will change to a different H3 platform in due course), and
displaying on a 1080p/60 monitor connected via hdmi.

My current gstreamer pipeline is gst-launch-1.0 -v udpsrc port=5600 !
"application/x-rtp, media=(string)video, clock-rate=(int)90000,
encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! h264parse !
v4l2slh264dec ! kmssink plane-id=31 plane-properties=s,zpos=3

if I change to fakesink, the decode is working beautifully (and according
to gstreamer latency stats is giving a decode time  of around 5ms - better
than I had hoped!) However I am having issues with kmssink. Initially I was
only getting 30fps display, but applying the kmssink fix for atomic drm
clients from
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/654 and
also the rockchip kernel fix I get smooth 60fps decoding, but only if I add
a ! queue ! element between  v4l2slh264dec and kmssink which pushes the
latency up unacceptably high.

Playing with ! queue ! output buffers (max-size-buffers=1 leaky=downstream)
shows it needs max-size-buffers=3 buffers before it displays smoothly. (at
max-size-buffers=1 I alternately 1 frame dropped then 2 frames dropped, at
max-size-buffers=2 it's one frame dropped then no frames dropped)

I presume this means kmssink is not sending back memory buffers fast enough
to be reused?
Can anyone set me on the right track to getting ultra-low latency decoding
working? For this application I don't need to worry about screen tearing or
similar artefacts, just getting this video on screen as soon as I possibly
can.

Thank you for any help
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20221121/2f219d1a/attachment.htm>