[RFC PATCH v2 00/18] Add DRM CRTC 3D LUT interface
Pekka Paalanen
ppaalanen at gmail.com
Fri Feb 10 09:28:46 UTC 2023
On Thu, 9 Feb 2023 13:27:02 -0100
Melissa Wen <mwen at igalia.com> wrote:
> On 01/31, Pekka Paalanen wrote:
> > On Mon, 9 Jan 2023 14:38:09 -0100
> > Melissa Wen <mwen at igalia.com> wrote:
> >
> > > On 01/09, Melissa Wen wrote:
> > > > Hi,
> > > >
> > > > After collecting comments in different places, here is a second version
> > > > of the work on adding DRM CRTC 3D LUT support to the current DRM color
> > > > mgmt interface. In comparison to previous proposals [1][2][3], here we
> > > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT,
> > > > that means the following DRM CRTC color correction pipeline:
> > > >
> > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT
> >
> > Hi Melissa,
> >
> > that makes sense to me, for CRTCs. It would be really good to have that
> > as a diagram in the KMS UAPI documentation.
> >
>
> Hi Pekka,
>
> Thanks for your feedbacks and your time reviewing this proposal.
No problem, and sorry it took so long!
I'm just finishing the catch-up with everything that happened during
winter holidays.
> > If someone wants to add a 3D LUT to KMS planes as well, then I'm not
> > sure if it should be this order or swapped. I will probably have an
> > opinion about that once Weston is fully HDR capable and has been tried
> > in the wild for a while with the HDR color operations fine-tuned based
> > on community feedback. IOW, not for a long time. The YUV to RGB
> > conversion factors in there as well.
> >
> I see, this is also the reason I reuse here Alex Hung's proposal for
> pre-blending API. I'll work on better documentation.
>
> >
> > > >
> > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung
> > > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a
> > > > LUT3D_SIZE, that allows userspace to use different supported settings of
> > > > 3D LUT, fitting VA-API and new color API better. In this sense, I
> > > > adjusted the pre-blending proposal for post-blending usage.
> > > >
> > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to
> > > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an
> > > > extra/optional patch to define a default value for LUT3D_MODE, inspired
> > > > by what we do for the plane blend mode property (pre-multiplied).
> > > >
> > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage
> > > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current
> > > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT
> > > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context
> > > > for DCN 301 (easily extendible to other DCN families). Finally, from
> > > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display
> > > > driver, exposing modes supported by HW and programming user shaper and
> > > > 3D LUT accordingly.
> > > >
> > > > Our target userspace is Gamescope/SteamOS.
> > > >
> > > > Basic IGT tests were based on [5][6] and are available here (in-progress):
> > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api
> > > >
> > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
> > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
> > > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/
> > > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/
> > > > [5] https://patchwork.freedesktop.org/series/90165/
> > > > [6] https://patchwork.freedesktop.org/series/109402/
> > > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html
> > > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11
> > > >
> > > > Let me know your thoughts.
> > >
> > > +Simon Ser, +Pekka Paalanen who might also be interested in this series.
> >
> > Unfortunately I don't have the patch emails to reply to, so here's a
> > messy bunch of comments. I'll concentrate on the UAPI design as always.
>
> Sorry, the patchset is here: https://lore.kernel.org/dri-devel/20230109143846.1966301-1-mwen@igalia.com/
> In the next version, I won't forget cc'ing you at first.
> >
> > +/*
> > + * struct drm_mode_lut3d_mode - 3D LUT mode information.
> > + * @lut_size: number of valid points on every dimension of 3D LUT.
> > + * @lut_stride: number of points on every dimension of 3D LUT.
> > + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher
> > + * bit_depth the least significant bits will be truncated.
> > + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616.
> > + * @flags: flags for hardware-sepcific features
> > + */
> > +struct drm_mode_lut3d_mode {
> > + __u16 lut_size;
> > + __u16 lut_stride[3];
> > + __u16 bit_depth;
> > + __u32 color_format;
> > + __u32 flags;
> > +};
> >
> > Why is lut_stride an array of 3, but lut_size is not?
>
> It cames from VA-API:
> https://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html#a682756be15d09327ba725b74a863cbcc
>
> In short, the reason is that lut_size is the valid points and is
> the same for every dimensions, but lut_stride may vary.
Ok, so lut_size is what I would have guessed it to be. Just needs to be
doc'd - pretty much all my questions are lack of docs.
I see that lut_stride is supposed to be in points, and not bytes. That
is surprising. Are you sure it's good?
> >
> > What is the color_mode the comment is referring to?
>
> It refers to FB color_mode/bpp. I'm not using it in post-blending 3D LUT
> implementation (should I?), it cames from pre-blending use case. Maybe
> the main issue here is if reusing the pre-blending 3D LUT mode struct is
> a good approach or better create a specific for post-blending.
Sorry, I have no idea what FB color_mode is. I do not recall any such
thing in the KMS uAPI.
> >
> > What is "number of bits of RGB"? Input precision? Output precision?
> > Integer or floating point?
>
> It's the bit depth of the 3D LUT values, the same for every channels. In
> the AMD case, it's supports 10-bit and 12-bit, for example.
Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any
hardware ever?
What exactly is the truncation the comment refers to?
It sounds like if input has higher precision than the LUT elements,
then "truncation" occurs. I can kind of see that, but I also think it
is a false characterisation. The LUT input precision affects the
precision of LUT indexing and the precision of interpolation between
the LUT elements. I would not expect those two precisions to be
truncated to the LUT element precision (but they could be truncated to
something else hardware specific). Instead, I do expect the
interpolation result to be truncated to the LUT output precision, which
probably is the same as the LUT element precision, but not necessarily.
Maybe the comment about truncation should simply be removed? The result
is obvious if we know the LUT input, element, and output precision, and
what exactly happens with the indexing and interpolation is probably
good enough to be left hardware-specific if it is difficult to describe
in generic terms across different hardware.
> >
> > Flags cannot be hardware specific, because it makes the whole KMS UAPI
> > hardware specific. That won't work. You have to have driver-agnostic
> > definitions for all possible flags.
> >
> > Why is this the whole first patch? There is no documentation for the
> > UAPI on how this struct works, so I cannot review this. Explaining just
> > the individual fields is not enough to understand it. Is this something
> > the kernel fills in and is read-only to userspace? Is userspace filling
> > this in?
>
> I see. I'll work on explaining/documenting it better.
> >
> >
> > + * “LUT3D”:
> > + * Blob property to set the 3D LUT mapping pixel data after the color
> > + * transformation matrix and before gamma 1D lut correction. The
> > + * data is interpreted as an array of &struct drm_color_lut elements.
> > + * Hardware might choose not to use the full precision of the LUT
> > + * elements.
> > + *
> > + * Setting this to NULL (blob property value set to 0) means a the output
> > + * color is identical to the input color. This is generally the driver
> > + * boot-up state too. Drivers can access this blob through
> > + * &drm_crtc_state.gamma_lut.
> > + *
> >
> > You need to define how the 1-D array of drm_color_lut elements blob
> > will be interpreted as a 3-D array for the 3D LUT, and how the
> > dimensions match to the R, G and B channels. It's a bit like the
> > question about row-major or column-major storage for matrices, except
> > more complicated and not in those words.
>
> ack
> >
> > + * “LUT3D_MODE”:
> > + * Enum property to give the mode of the 3D lookup table to be set on the
> > + * LUT3D property. A mode specifies size, stride, bit depth and color
> > + * format and depends on the underlying hardware). If drivers support
> > + * multiple 3D LUT modes, they should be declared in a array of
> > + * drm_color_lut3d_mode and they will be advertised as an enum.
> >
> > How does that work exactly? I didn't get it. I could guess, but having
> > to guess on API is bad.
>
> The driver advertises all supported modes (each combination of values)
> in a array as a enum, userspace can check all accepted modes and set the
> one that fits the user 3D LUT settings. I think it's possible to get the
> idea from this IGT test:
> https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/8771f444c3dcd126d7590d5a9b1b0db9706bbf6e#ed5dbc960ac210e3fbacd2361fe0270709767aaa_205_205
> >
You lost me at "an array as an enum".
I understand there is a blob containing an array of struct
drm_mode_lut3d_mode. What I don't understand is that you say LUT3D_MODE
is an enum property. Where does the blob come from, then? What property
provides the blob?
Am I correct in guessing that the values of LUT3D_MODE enum property
are indices into the array in the blob, and that userspace will set it?
That sounds good to me, if it's the integer value of the enum. But enum
values also need string names, because that is how the values are
usually recognized, so what name strings will be used?
In that code example, I cannot see any connection between the array of
drm_mode_lut3d_mode entries, the 'name' string, and the actual data
generated for the LUT. They must all connect somehow, but it just
doesn't seem to be in the code. It is just hardcoding mode_lut3d[0],
and the data generator never sees even that - it gets the lut size, but
none of the other parameters. I cannot see that working across multiple
drivers.
Why is it setting a linear shaper LUT explicitly instead of unsetting
it? I mean this preparation:
shaper_linear = generate_table(data->shaper_lut_size, 1.0);
"Linear" actually means "identity", right? That's what people usually
write when they mean identity.
Surely it's not actually a constant curve? That would make the whole
test useless for proving the 3D LUT works.
> >
> > + /**
> > + * @lut3d:
> > + *
> > + * 3D Lookup table for converting pixel data. Position where it takes
> > + * place depends on hw design, after @ctm or @gamma_lut. See
> > + * drm_crtc_enable_color_mgmt(). The blob (if not NULL) is an array of
> > + * &struct drm_color_lut.
> > + */
> > + struct drm_property_blob *lut3d;
> >
> > I do not like the wording of "depends on hw design", and it is used in
> > very many places here. The KMS UAPI semantics cannot vary based on
> > hardware. Your cover letter defines the order in the color pipeline, so
> > I don't understand how this here can depend on hw.
> >
> > What can depend on hardware is which KMS UAPI properties are exposed,
> > and how you map a property to a hardware unit (which can even change
> > based on the exact pipeline configuration as long as the results are as
> > the UAPI doc defines). But this comment here is talking about the UAPI
> > properties, not hw elements.
> >
>
> You are right! My initial idea was to explain that it's possible for
> other vendors color pipeline to fit this pipeline internally, if they
> need a 1D LUT before the 3D LUT, but not the 1D LUT in the end.
>
> >
> > I'm happy that the 3D LUT interface is being developed, but as you can
> > see from my questions, the UAPI documentation is practically missing. I
> > would have no idea how to use this as is.
>
> Thank you again for your valuable comments. I'll address your comments
> in a next version by better explaining all these points.
Thank you for working on this!
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/dri-devel/attachments/20230210/4ab7f5d2/attachment.sig>
More information about the dri-devel
mailing list