Overlay support in the i.MX7 display

[Nicolas Dufresne]

Le mardi 05 novembre 2019 à 10:17 +0100, Philipp Zabel a écrit :
> Hi Laurent,
> 
> On Fri, 2019-11-01 at 10:43 +0200, Laurent Pinchart wrote:
> > Hello,
> > 
> > I'm looking at the available options to support overlays in the display
> > pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support
> > overlays, the feature being implemented in the PXP. A driver for the PXP
> > is available but only supports older SoCs whose PXP doesn't support
> > overlays. This driver is implemented as a V4L2 mem2mem driver, which
> > makes support of additional input channels impossible.
> > 
> > Here are the options I can envision:
> > 
> > - Extend the existing PXP driver to support multiple channels. This is
> >   technically feasible, but will require moving away from the V4L2
> >   mem2mem framework, which would break userspace. I don't think this
> >   path could lead anywhere.
> 
> I may be biased, but please don't break the V4L2 mem2mem usecase :)
> 
> > - Write a new PXP driver for the i.MX7, still using V4L2, but with
> >   multiple video nodes. This would allow blending multiple layers, but
> >   would require writing the output to memory, while the PXP has support
> >   for direct connections to the LCDIF (through small SRAM buffers).
> >   Performances would thus be suboptimal. The API would also be awkward,
> >   as using the PXP for display would require usage of V4L2 in
> >   applications.
> 
> I'm not sure V4L2 is the best API for multi-pass 2D composition,
> especially as the PXP is able to blit an overlay onto a background in
> place.

There was some userspace (GStreamer element) doing exactly that with
v4l2 m2m using the imx6 driver. The API was fine, even though fences
would have made programming it easier.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/308

(not merge as we don't have an agreement on kernel side, notably we
don't even have a way to control the blend function, so the result is
likely dependant on the use case the driver was written for)

The real limitation was that these IP usually supports more then just
blit/blend over another surface, and as you said, supports background.
And to support this use case, we'd need an m2m driver with multiple queues per direction. That was discussed in that last workshop at ELCE, and applies to other m2m IP like muxers and demuxers which exist on STB kind of SoC.

> 
> > - Extend the mxsfb driver with PXP support, and expose the PXP inputs as
> >   KMS planes. The PXP would only be used when available, and would be
> >   transparent to applications. This would however prevent using it
> >   separately from the display (to perform multi-pass alpha blending for
> >   instance).
> 
> For the SRAM block row buffer path to the LCDIF, I think the KMS plane
> abstraction is the way to go. The DRM and V4L2 drivers could be made to
> use a shared backend, such that only one of plane composition and V4L2
> scaling/CSC functions can work at the same time.
> 
> > What would be the best option going forward ? Would any of you, by any
> > chance, have already started work in this area ?
> 
> I have not worked on this.
> 
> regards
> Philipp
>