[RFC] Plane color pipeline KMS uAPI

[Joshua Ashton]

When we are talking about being 'prescriptive' in the API, are we
outright saying we don't want to support arbitrary 3D LUTs, or are we
just offering certain algorithms to be 'executed' for a plane/crtc/etc
in the atomic API? I am confused...

There is so much stuff to do with color, that I don't think a
prescriptive API in the kernel could ever keep up with the things that
we want to be pushing from Gamescope/SteamOS. For example, we have so
many things going on, night mode, SDR gamut widening, HDR/SDR gain,
the ability to apply 'looks' for eg. invert luma or for retro looks,
enhanced contrast, tonemapping, inverse tonemapping... We also are
going to be doing a bunch of stuff with EETFs for handling out of
range HDR content for scanout.

Some of what we do is kinda standard, regular "there is a paper on
this" algorithms, and others are not.
While yes, it might be very possible to do simple things, once you
start wanting to do something 'different', that's kinda lock-in.

Whether this co-exists with arbitrary LUTs (that we definitely want
for SteamOS) or not:
I think putting a bunch of math-y stuff like this into the kernel is
probably the complete wrong approach. Everything would need to be
fixed point and it would be a huge pain in the butt to deal with on
that side.

Maybe this is a "hot take", but IMO, DRM atomic is already waaay too
much being done in the kernel space. I think making it go even further
and having it be a prescriptive color API is a complete step in the
wrong direction.

There is also the problem of... if there is a bug in the math here or
we want to add a new feature, if it's kernel side, you are locked in
to having that bug until the next release on your distro and probably
years if it's a new feature!
Updating kernels is much harder for 'enterprise' distros if it is not
mission critical. Having all of this in userspace is completely fine
however...

If you want to make some userspace prescriptive -> descriptive color
library I am all for that for general case compositors, but I don't
think I would use something like that in Gamescope.
That's not to be rude, we are just picky and want freedom to do what
we want and iterate on it easily.

I guess this all comes back to my initial point... having some
userspace to handle stuff that is either kinda or entirely vendor
specific is the right way of solving this problem :-P

- Joshie 🐸✨

On Thu, 11 May 2023 at 09:51, Karol Herbst <kherbst at redhat.com> wrote:
>
> On Wed, May 10, 2023 at 9:59 AM Jonas Ådahl <jadahl at redhat.com> wrote:
> >
> > On Tue, May 09, 2023 at 08:22:30PM +0000, Simon Ser wrote:
> > > On Tuesday, May 9th, 2023 at 21:53, Dave Airlie <airlied at gmail.com> wrote:
> > >
> > > > There are also other vendor side effects to having this in userspace.
> > > >
> > > > Will the library have a loader?
> > > > Will it allow proprietary plugins?
> > > > Will it allow proprietary reimplementations?
> > > > What will happen when a vendor wants distros to ship their
> > > > proprietary fork of said library?
> > > >
> > > > How would NVIDIA integrate this with their proprietary stack?
> > >
> > > Since all color operations exposed by KMS are standard, the library
> > > would just be a simple one: no loader, no plugin, no proprietary pieces,
> > > etc.
> > >
> >
> > There might be pipelines/color-ops only exposed by proprietary out of
> > tree drivers; the operation types and semantics should ideally be
> > defined upstream, but the code paths would in practice be vendor
> > specific, potentially without any upstream driver using them. It should
> > be clear whether an implementation that makes such a pipeline work is in
> > scope for the upstream library.
> >
> > The same applies to the kernel; it must be clear whether pipeline
> > elements that potentially will only be exposed by out of tree drivers
> > will be acceptable upstream, at least as documented operations.
> >
>
> they aren't. All code in the kernel needs to be used by in-tree
> drivers otherwise it's fair to delete it. DRM requires any UAPI change
> to have a real open source user in space user.
>
> Nvidia knows this and they went to great lengths to fulfill this
> requirement in the past. They'll manage.
>
> >
> > Jonas
> >
>