kmssink and ultra low latency

[Nirbheek Chauhan]

On Tue, Nov 22, 2022 at 12:30 AM Nicolas Dufresne via gstreamer-devel
<gstreamer-devel at lists.freedesktop.org> wrote:
> Le lundi 21 novembre 2022 à 13:24 +0000, Peter Allen via gstreamer-devel a
> écrit :
> > I have an application that needs ultra-low latency decoding and display of a
> > 720p/60 H264 rtp stream. I'm using Allwinner H3 (currently an Orange Pi PC,
> > but that will change to a different H3 platform in due course), and displaying
> > on a 1080p/60 monitor connected via hdmi.
> >
> > My current gstreamer pipeline is gst-launch-1.0 -v udpsrc port=5600 !
> > "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-
> > name=(string)H264, payload=(int)96" ! rtph264depay ! h264parse ! v4l2slh264dec
> > ! kmssink plane-id=31 plane-properties=s,zpos=3
> >
> > if I change to fakesink, the decode is working beautifully (and according to
> > gstreamer latency stats is giving a decode time  of around 5ms - better than I
> > had hoped!) However I am having issues with kmssink. Initially I was only
> > getting 30fps display, but applying the kmssink fix for atomic drm clients
> > from https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/654 and
> > also the rockchip kernel fix I get smooth 60fps decoding, but only if I add a
> > ! queue ! element between  v4l2slh264dec and kmssink which pushes the latency
> > up unacceptably high.
> >
> > Playing with ! queue ! output buffers (max-size-buffers=1 leaky=downstream)
> > shows it needs max-size-buffers=3 buffers before it displays smoothly. (at
> > max-size-buffers=1 I alternately 1 frame dropped then 2 frames dropped, at
> > max-size-buffers=2 it's one frame dropped then no frames dropped)
>
> I can only presume that render time correctness does not matter to you. So my
> advise would be to use kmssink sync=0.

Another thing to try before turning off sync is changing
processing-deadline to a lower value. Honestly the default 20ms is a
bit too large.