[RFC 02/33] drm: Add color operation structure
Pekka Paalanen
ppaalanen at gmail.com
Tue Sep 5 11:33:05 UTC 2023
On Mon, 4 Sep 2023 14:10:05 +0000
"Shankar, Uma" <uma.shankar at intel.com> wrote:
> > -----Original Message-----
> > From: dri-devel <dri-devel-bounces at lists.freedesktop.org> On Behalf Of Pekka
> > Paalanen
> > Sent: Wednesday, August 30, 2023 6:30 PM
> > To: Shankar, Uma <uma.shankar at intel.com>
> > Cc: intel-gfx at lists.freedesktop.org; Borah, Chaitanya Kumar
> > <chaitanya.kumar.borah at intel.com>; dri-devel at lists.freedesktop.org; wayland-
> > devel at lists.freedesktop.org
> > Subject: Re: [RFC 02/33] drm: Add color operation structure
> >
> > On Tue, 29 Aug 2023 21:33:51 +0530
> > Uma Shankar <uma.shankar at intel.com> wrote:
> >
> > > From: Chaitanya Kumar Borah <chaitanya.kumar.borah at intel.com>
> > >
> > > Each Color Hardware block will be represented uniquely in the color
> > > pipeline. Define the structure to represent the same.
> > >
> > > These color operations will form the building blocks of a color
> > > pipeline which best represents the underlying Hardware. Color
> > > operations can be re-arranged, substracted or added to create distinct
> > > color pipelines to accurately describe the Hardware blocks present in
> > > the display engine.
> > >
> > > Co-developed-by: Uma Shankar <uma.shankar at intel.com>
> > > Signed-off-by: Uma Shankar <uma.shankar at intel.com>
> > > Signed-off-by: Chaitanya Kumar Borah <chaitanya.kumar.borah at intel.com>
> > > ---
> > > include/uapi/drm/drm_mode.h | 72
> > > +++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 72 insertions(+)
> > >
> > > diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
> > > index ea1b639bcb28..882479f41745 100644
> > > --- a/include/uapi/drm/drm_mode.h
> > > +++ b/include/uapi/drm/drm_mode.h
> > > @@ -943,6 +943,78 @@ struct hdr_output_metadata {
> > > };
> > > };
> > >
> > > +/**
> > > + * enum color_op_block
> > > + *
> > > + * Enums to identify hardware color blocks.
> > > + *
> > > + * @DRM_CB_PRE_CSC: LUT before the CTM unit
> > > + * @DRM_CB_CSC: CTM hardware supporting 3x3 matrix
> > > + * @DRM_CB_POST_CSC: LUT after the CTM unit
> > > + * @DRM_CB_3D_LUT: LUT hardware with coefficients for all
> > > + * color components
> > > + * @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor
> > > + * can expose a custom hardware by defining a
> > > + * color operation block with this name as
> > > + * identifier
> >
> > This naming scheme does not seem to work. It assumes a far too rigid pipeline,
> > just like the old KMS property design. What if you have two other operations
> > between PRE_CSC and CSC?
> >
> > What sense do PRE_CSC and POST_CSC make if you don't happen to have a CSC
> > operation?
>
> Sure, we can re-look at the naming. However, it will be good to define some standard
> operations common to all vendors and keep the rest as vendor private.
My opinion is that these "names" should not even be an enum at all.
You're talking about operations, but operation is 'type', not 'name'.
There needs to be no pre-defined, hardcoded pipeline structure in the
UAPI design. Your naming scheme, and the old KMS color property design,
assume that there is such a rigid pre-defined structure that all
hardware and drivers must adhere to. That does not work.
Drivers need to be able to expose any arbitrary operations in any
arbitrary order in each pipeline, with no pre-determined or hinted
"intended use". We want to define the operations, and not say what they
should be used for. Usage examples are for accompanying documentation,
not API tokens.
This idea is at the core of the new color pipeline design we have been
discussing.
> > What if a driver put POST_CSC before PRE_CSC in its pipeline?
> >
> > What if your CSC is actually a series of three independent operations, and in
> > addition you have PRE_CSC and POST_CSC?
>
> We should try to standardized the operations as much as possible and leave rest as
> vendor private. Current proposal allows us to do that.
>
> > 3D_LUT is an operation category, not a name. The same could be said about
> > private.
>
> Sure, will fix this.
>
> > Given that all these are also UAPI, do we also need protect old userspace from
> > seeing values it does not understand?
>
> For the values userspace doesn't understand, it can ignore the blocks. We should ensure
> that userspace always gets a clean state wrt color hardware state and no baggage from
> another client should be there. With that there is no burden of disabling that particular
> block will be there on an older userspace.
>
> > > + */
> > > +enum color_op_block {
> > > + DRM_CB_INVAL = -1,
> > > +
> > > + DRM_CB_PRE_CSC = 0,
> > > + DRM_CB_CSC,
> > > + DRM_CB_POST_CSC,
> > > + DRM_CB_3D_LUT,
> > > +
> > > + /* Any new generic hardware block can be updated here */
> > > +
> > > + /*
> > > + * PRIVATE is kept at 255 to make it future proof and leave
> > > + * scope for any new addition
> > > + */
> > > + DRM_CB_PRIVATE = 255,
> > > + DRM_CB_MAX = DRM_CB_PRIVATE,
> > > +};
> > > +
> > > +/**
> > > + * enum color_op_type
> > > + *
> > > + * These enums are to identify the mathematical operation that
> > > + * a hardware block is capable of.
> > > + * @CURVE_1D: It represents a one dimensional lookup table
> > > + * @CURVE_3D: Represents lut value for each color component for 3d
> > > +lut capable hardware
> > > + * @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware
> > > + * @FIXED_FUNCTION: To enable and program any custom fixed function
> > > +hardware unit */ enum color_op_type {
> > > + CURVE_1D,
> > > + CURVE_3D,
> > > + MATRIX,
> > > + FIXED_FUNCTION,
> >
> > My assumption was that a color_op_type would clearly and uniquely define the
> > mathematical model of the operation and the UABI structure of the parameter
> > blob. That means we need different values for uniform vs. exponentially vs.
> > programmable distributed 1D LUT, etc.
>
> In the hardware the LUTS are programmed as they are received from userspace.
> So once the userspace gets to know the distribution of LUTS, segments, precision,
> Number of lut samples, it can create the lut values to be programmed.
>
> This information on the distribution of luts in the hardware can be extracted by the
> drm_color_lut_range structure which is exposed as blob in the hardware block with
> TYPE set as CURVE_1D.
I was referring to the special cases like FIXED_FUNCTION and PRIVATE.
If you have an operation of type PRIVATE, you have no idea what it does
until you decipher 'private_flags', but you cannot decipher
'private_flags' until you know the driver name because it is
vendor-specific.
Userspace hates keying things off driver name. It means no other driver
can expose the same and expect userspace to just start using it. It
prevents an originally single-vendor feature to become a common
multi-vendor feature in future hardware generations without patching
every KMS client. Think about OpenGL or Vulkan: they have
vendor-specific extensions, but if another vendor implements the same,
they don't make up another extension but claim to support the original
one.
> > If there is a 1D curve with pre-programmed (fixed and named) curves, we need to
> > enumerate all the curve types somehow. Probably each fixed curve type should
> > not be a different operation type, because that would explode the number of
> > alternative pipelines.
>
> Exposing the lut distribution with drm_color_lut_range would do this job.
I'm talking about named fixed curves, PQ EOTF for example. You'd
probably put it in FIXED_FUNCTION.
Or are you suggesting that userspace should take the LUT
parameterisation from the driver and check if it happens to match PQ
EOTF close enough?
We really need (string) names for fixed curves, so that they can be
readily identified. Maybe Intel does not have them then, but AMD has.
> > A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you meant a 3D
> > LUT which is a {x,y,z} = f(t,u,v) - a 3-vector field in three dimensional space.
>
> Yes right, we can optimize and fine tune this.
I mean you used a wrong term. It's not a 3D curve when you have a 3D
LUT.
> > A matrix element could be with or without an offset vector I guess.
> >
> > FIXED_FUNCTION would need to be replaced with e.g. your example
> > VENDORXXX_BT602_TO_BT2020 to work.
> >
> > Have I missed something, how did you intend this to work?
>
> FIXED_FUNCTION is for some descriptive implementation. Some cases where
> Hardware just have a bit to enable and rest of the implementation is within hardware.
> Since type is used for other general hardware blocks, its not intended to cover vendor
> specific implementation and a private_flags are used instead.
As I've said, 'private_flags' cannot exist.
Thanks,
pq
>
> Thanks Pekka for the feedback.
>
> Regards,
> Uma Shankar
>
> >
> >
> > Thanks,
> > pq
> >
> > > +};
> > > +
> > > +/**
> > > + * @struct drm_color_op
> > > + *
> > > + * This structure is used to represent the capability of
> > > + * individual color hardware blocks.
> > > + *
> > > + * @name: a standardized enum to identify the color hardware block
> > > + * @type: The type of mathematical operation it can perform
> > > + * @blob_id: Id pointing to a blob containing information about
> > > + * the hardware block which advertizes its capabilities
> > > + * to the userspace. It can be an optional field depending
> > > + * on the members "name" and "type".
> > > + * @private_flags: This can be used to provide vendor specific hints
> > > + * to user space
> > > + */
> > > +struct drm_color_op {
> > > + enum color_op_block name;
> > > + enum color_op_type type;
> > > + __u32 blob_id;
> > > + __u32 private_flags;
> > > +};
> > > +
> > > /**
> > > * DRM_MODE_PAGE_FLIP_EVENT
> > > *
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20230905/7120785c/attachment-0001.sig>
More information about the wayland-devel
mailing list