[RFC] Plane color pipeline KMS uAPI

[Jonas Ådahl]

On Thu, May 11, 2023 at 04:56:47PM +0000, Joshua Ashton wrote:
> When we are talking about being 'prescriptive' in the API, are we
> outright saying we don't want to support arbitrary 3D LUTs, or are we
> just offering certain algorithms to be 'executed' for a plane/crtc/etc
> in the atomic API? I am confused...

The 'prescriptive' idea that the RFC of this thread proposes *is* a way
to support arbitrary 3D LUTs (and other mathematical operations),
arbitrarily, in a somewhat vendored way, only that it will not be vendor
prefixed hard coded properties with specific positions in the pipeline,
but instead more or less an introspectable pipeline, describing what
kind of LUT's, Matrix multiplication (and in what order) etc a hardware
can do.

The theoretical userspace library would be the one turning descriptive
"please turn this into that" requests into the "prescriptive" color
pipeline operations. It would target general purpose compositors, but it
wouldn't be mandatory. Doing vendor specific implemantions in gamescope
would be possible; it wouldn't look like the verion that exist somewhere
now that uses a bunch of AMD_* properties, it'd look more like the
example Simon had in the initial RFC.


Jonas

> 
> There is so much stuff to do with color, that I don't think a
> prescriptive API in the kernel could ever keep up with the things that
> we want to be pushing from Gamescope/SteamOS. For example, we have so
> many things going on, night mode, SDR gamut widening, HDR/SDR gain,
> the ability to apply 'looks' for eg. invert luma or for retro looks,
> enhanced contrast, tonemapping, inverse tonemapping... We also are
> going to be doing a bunch of stuff with EETFs for handling out of
> range HDR content for scanout.
> 
> Some of what we do is kinda standard, regular "there is a paper on
> this" algorithms, and others are not.
> While yes, it might be very possible to do simple things, once you
> start wanting to do something 'different', that's kinda lock-in.
> 
> Whether this co-exists with arbitrary LUTs (that we definitely want
> for SteamOS) or not:
> I think putting a bunch of math-y stuff like this into the kernel is
> probably the complete wrong approach. Everything would need to be
> fixed point and it would be a huge pain in the butt to deal with on
> that side.
> 
> Maybe this is a "hot take", but IMO, DRM atomic is already waaay too
> much being done in the kernel space. I think making it go even further
> and having it be a prescriptive color API is a complete step in the
> wrong direction.
> 
> There is also the problem of... if there is a bug in the math here or
> we want to add a new feature, if it's kernel side, you are locked in
> to having that bug until the next release on your distro and probably
> years if it's a new feature!
> Updating kernels is much harder for 'enterprise' distros if it is not
> mission critical. Having all of this in userspace is completely fine
> however...
> 
> If you want to make some userspace prescriptive -> descriptive color
> library I am all for that for general case compositors, but I don't
> think I would use something like that in Gamescope.
> That's not to be rude, we are just picky and want freedom to do what
> we want and iterate on it easily.
> 
> I guess this all comes back to my initial point... having some
> userspace to handle stuff that is either kinda or entirely vendor
> specific is the right way of solving this problem :-P
> 
> - Joshie 🐸✨
> 
> On Thu, 11 May 2023 at 09:51, Karol Herbst <kherbst at redhat.com> wrote:
> >
> > On Wed, May 10, 2023 at 9:59 AM Jonas Ådahl <jadahl at redhat.com> wrote:
> > >
> > > On Tue, May 09, 2023 at 08:22:30PM +0000, Simon Ser wrote:
> > > > On Tuesday, May 9th, 2023 at 21:53, Dave Airlie <airlied at gmail.com> wrote:
> > > >
> > > > > There are also other vendor side effects to having this in userspace.
> > > > >
> > > > > Will the library have a loader?
> > > > > Will it allow proprietary plugins?
> > > > > Will it allow proprietary reimplementations?
> > > > > What will happen when a vendor wants distros to ship their
> > > > > proprietary fork of said library?
> > > > >
> > > > > How would NVIDIA integrate this with their proprietary stack?
> > > >
> > > > Since all color operations exposed by KMS are standard, the library
> > > > would just be a simple one: no loader, no plugin, no proprietary pieces,
> > > > etc.
> > > >
> > >
> > > There might be pipelines/color-ops only exposed by proprietary out of
> > > tree drivers; the operation types and semantics should ideally be
> > > defined upstream, but the code paths would in practice be vendor
> > > specific, potentially without any upstream driver using them. It should
> > > be clear whether an implementation that makes such a pipeline work is in
> > > scope for the upstream library.
> > >
> > > The same applies to the kernel; it must be clear whether pipeline
> > > elements that potentially will only be exposed by out of tree drivers
> > > will be acceptable upstream, at least as documented operations.
> > >
> >
> > they aren't. All code in the kernel needs to be used by in-tree
> > drivers otherwise it's fair to delete it. DRM requires any UAPI change
> > to have a real open source user in space user.
> >
> > Nvidia knows this and they went to great lengths to fulfill this
> > requirement in the past. They'll manage.
> >
> > >
> > > Jonas
> > >
> >
>