[Intel-gfx] [RFC v2 00/22] Add Support for Plane Color Lut and CSC features

[Harry Wentland]

On 2021-10-12 17:01, Shankar, Uma wrote:
> 
> 
>> -----Original Message-----
>> From: Pekka Paalanen <ppaalanen at gmail.com>
>> Sent: Tuesday, October 12, 2021 5:25 PM
>> To: Shankar, Uma <uma.shankar at intel.com>
>> Cc: intel-gfx at lists.freedesktop.org; dri-devel at lists.freedesktop.org;
>> harry.wentland at amd.com; ville.syrjala at linux.intel.com; brian.starkey at arm.com;
>> sebastian at sebastianwick.net; Shashank.Sharma at amd.com
>> Subject: Re: [RFC v2 00/22] Add Support for Plane Color Lut and CSC features
>>
>> On Tue,  7 Sep 2021 03:08:42 +0530
>> Uma Shankar <uma.shankar at intel.com> wrote:
>>
>>> This is how a typical display color hardware pipeline looks like:
>>>  +-------------------------------------------+
>>>  |                RAM                        |
>>>  |  +------+    +---------+    +---------+   |
>>>  |  | FB 1 |    |  FB 2   |    | FB N    |   |
>>>  |  +------+    +---------+    +---------+   |
>>>  +-------------------------------------------+
>>>        |  Plane Color Hardware Block |
>>> +--------------------------------------------+
>>>  | +---v-----+   +---v-------+   +---v------+ |
>>>  | | Plane A |   | Plane B   |   | Plane N  | |
>>>  | | DeGamma |   | Degamma   |   | Degamma  | |
>>>  | +---+-----+   +---+-------+   +---+------+ |
>>>  |     |             |               |        |
>>>  | +---v-----+   +---v-------+   +---v------+ |
>>>  | |Plane A  |   | Plane B   |   | Plane N  | |
>>>  | |CSC/CTM  |   | CSC/CTM   |   | CSC/CTM  | |
>>>  | +---+-----+   +----+------+   +----+-----+ |
>>>  |     |              |               |       |
>>>  | +---v-----+   +----v------+   +----v-----+ |
>>>  | | Plane A |   | Plane B   |   | Plane N  | |
>>>  | | Gamma   |   | Gamma     |   | Gamma    | |
>>>  | +---+-----+   +----+------+   +----+-----+ |
>>>  |     |              |               |       |
>>>  +--------------------------------------------+
>>> +------v--------------v---------------v-------|
>>> ||                                           ||
>>> ||           Pipe Blender                    ||
>>> +--------------------+------------------------+
>>> |                    |                        |
>>> |        +-----------v----------+             |
>>> |        |  Pipe DeGamma        |             |
>>> |        |                      |             |
>>> |        +-----------+----------+             |
>>> |                    |            Pipe Color  |
>>> |        +-----------v----------+ Hardware    |
>>> |        |  Pipe CSC/CTM        |             |
>>> |        |                      |             |
>>> |        +-----------+----------+             |
>>> |                    |                        |
>>> |        +-----------v----------+             |
>>> |        |  Pipe Gamma          |             |
>>> |        |                      |             |
>>> |        +-----------+----------+             |
>>> |                    |                        |
>>> +---------------------------------------------+
>>>                      |
>>>                      v
>>>                Pipe Output
>>>
>>> This patch series adds properties for plane color features. It adds
>>> properties for degamma used to linearize data and CSC used for gamut
>>> conversion. It also includes Gamma support used to again non-linearize
>>> data as per panel supported color space. These can be utilize by user
>>> space to convert planes from one format to another, one color space to
>>> another etc.
>>>
>>> Userspace can take smart blending decisions and utilize these hardware
>>> supported plane color features to get accurate color profile. The same
>>> can help in consistent color quality from source to panel taking
>>> advantage of advanced color features in hardware.
>>>
>>> These patches add the property interfaces and enable helper functions.
>>> This series adds Intel's XE_LPD hw specific plane gamma feature. We
>>> can build up and add other platform/hardware specific implementation
>>> on top of this series.
>>>
>>> Credits: Special mention and credits to Ville Syrjala for coming up
>>> with a design for this feature and inputs. This series is based on his
>>> original design and idea.
>>>
>>> Note: Userspace support for this new UAPI will be done on Chrome in
>>> alignment with weston and general opensource community.
>>> Discussion ongoing with Harry Wentland, Pekka and community on color
>>> pipeline and UAPI design. Harry's RFC below:
>>> https://patchwork.freedesktop.org/series/89506/>>>> We need to converge on a common UAPI interface which caters to all the
>>> modern color hardware pipelines.
>>>
>>> ToDo: State readout for this feature will be added next.
>>>
>>> v2: Added UAPI description and added change in the rfc section of
>>> kernel Documentation folder
>>
>> Hi,
>>
>> thank you for this. I do believe the KMS UAPI should expose what hardware can do
>> (prescribed operations) rather than how they would be often used (to realize a
>> conversion from one space description to another). This proposal fits quite nicely
>> with what I have envisioned for Weston.
> 

It's taken me a while but I am starting to agree with the prescriptive approach to
expose HW functionality. One thing we'll want to be careful of is to make sure
this isn't tied to specific HW more than it needs to be. I'll comment in other
places of this patchset to elaborate.

What's making me come around, i.e. change from a prescriptive (these are the
input/output/blending spaces/formats) to a descriptive (these are the LUTs and
CTMs) approach?

1) The prescriptive way has no good way of dealing with gamut and tone mapping.
   To do so we would need explicit OOTFs and CTMs or 3D LUTs anyways.

2) The prescriptive way provides no semblance of guarantee that transforms
   are equivalent when the compositor uses shaders transforms and composition
   vs when the compositor uses KMS transforms and composition.

3) Policy about treatment of surfaces/planes and blending is best left with
   the compositor (for the above reasons).

>> I mainly went over the big picture by commenting in detail on the proposal
>> document, and not looking too carefully at the other documentation or UAPI details
>> at this time.
> 
> Thanks Pekka for the feedback.
> 
>> Unfortunately I was unable to decipher how userspace is supposed to use the
>> XE_LPD special gamma features.
> 
> I will include the details on how userspace should actually get this through a sample
> IGT reference, that should help make this clear.
> 

It looks like with your current definitions each userspace compositor (Weston, kwin,
mutter, wlroots, Chrome's compositor, Android's compositor, etc.) would need to learn
how to program the XE_LPD LUTs as well as AMD LUTs. Would these definitions change
in future Intel HW generations? Would this mean all compositors would need to learn
again how to program the future LUT format?

Other options would be to give userspace a generic LUT with 4k FP16 entries and then
re-map that to the HW LUT in the kernel driver.

I might be missing some of the nuances of the XE_LPD LUT but it seems to me that the
main difference between different PWL implementations is the distribution of the
points used to define the LUT. Maybe a more generic PWL implementation could have
a kernel driver report one (or more) PWL point distributions. We could encode these
as enums and pre-defined arrays in a UAPI header. That way the compositor could have
a single, generic implementation of programming PWL in FP16 and the kernel driver
would only need to remap the FP16 to the HW-internal format, which is a trivial
conversion. Using this approach compositors would implement PWL support once and won't
have to touch it again in the future. Is there anything that would make this approach
a bad idea for Intel HW (or other HW)? (Credit for this idea goes to Vitaly)

Harry

> Regards,
> Uma Shankar
> 
>>
>> Thanks,
>> pq
>>
>>>
>>> Uma Shankar (22):
>>>   drm: RFC for Plane Color Hardware Pipeline
>>>   drm: Add Enhanced Gamma and color lut range attributes
>>>   drm: Add Plane Degamma Mode property
>>>   drm: Add Plane Degamma Lut property
>>>   drm/i915/xelpd: Define Degamma Lut range struct for HDR planes
>>>   drm/i915/xelpd: Add register definitions for Plane Degamma
>>>   drm/i915/xelpd: Enable plane color features
>>>   drm/i915/xelpd: Add color capabilities of SDR planes
>>>   drm/i915/xelpd: Program Plane Degamma Registers
>>>   drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
>>>   drm/i915/xelpd: Initialize plane color features
>>>   drm/i915/xelpd: Load plane color luts from atomic flip
>>>   drm: Add Plane CTM property
>>>   drm: Add helper to attach Plane ctm property
>>>   drm/i915/xelpd: Define Plane CSC Registers
>>>   drm/i915/xelpd: Enable Plane CSC
>>>   drm: Add Plane Gamma Mode property
>>>   drm: Add Plane Gamma Lut property
>>>   drm/i915/xelpd: Define and Initialize Plane Gamma Lut range
>>>   drm/i915/xelpd: Add register definitions for Plane Gamma
>>>   drm/i915/xelpd: Program Plane Gamma Registers
>>>   drm/i915/xelpd: Enable plane gamma
>>>
>>>  Documentation/gpu/drm-kms.rst                 |  90 +++
>>>  Documentation/gpu/rfc/drm_color_pipeline.rst  | 167 ++++++
>>>  drivers/gpu/drm/drm_atomic.c                  |   1 +
>>>  drivers/gpu/drm/drm_atomic_state_helper.c     |  12 +
>>>  drivers/gpu/drm/drm_atomic_uapi.c             |  38 ++
>>>  drivers/gpu/drm/drm_color_mgmt.c              | 177 +++++-
>>>  .../gpu/drm/i915/display/intel_atomic_plane.c |   6 +
>>>  .../gpu/drm/i915/display/intel_atomic_plane.h |   2 +
>>>  drivers/gpu/drm/i915/display/intel_color.c    | 513 ++++++++++++++++++
>>>  drivers/gpu/drm/i915/display/intel_color.h    |   2 +
>>>  .../drm/i915/display/skl_universal_plane.c    |  15 +-
>>>  drivers/gpu/drm/i915/i915_drv.h               |   3 +
>>>  drivers/gpu/drm/i915/i915_reg.h               | 176 +++++-
>>>  include/drm/drm_mode_object.h                 |   2 +-
>>>  include/drm/drm_plane.h                       |  81 +++
>>>  include/uapi/drm/drm_mode.h                   |  58 ++
>>>  16 files changed, 1337 insertions(+), 6 deletions(-)  create mode
>>> 100644 Documentation/gpu/rfc/drm_color_pipeline.rst
>>>
>