[PATCH V8 03/43] drm/doc/rfc: Describe why prescriptive color pipeline is needed

[Simon Ser]

Thanks a lot for these docs, very well written. I especially like the
"Driver Implementer's Guide" section.

A few minor comments below, regardless:

Reviewed-by: Simon Ser <contact at emersion.fr>

> +What problem are we solving?
> +============================
> +
> +We would like to support pre-, and post-blending complex color
> +transformations in display controller hardware in order to allow for
> +HW-supported HDR use-cases, as well as to provide support to
> +color-managed applications, such as video or image editors.
> +
> +It is possible to support an HDR output on HW supporting the Colorspace
> +and HDR Metadata drm_connector properties, but that requires the
> +compositor or application to render and compose the content into one
> +final buffer intended for display. Doing so is costly.
> +
> +Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and other
> +operations to support color transformations. These operations are often
> +implemented in fixed-function HW and therefore much more power efficient than
> +performing similar operations via shaders or CPU.
> +
> +We would like to make use of this HW functionality to support complex color
> +transformations with no, or minimal CPU or shader load.

I would also highlight that we need to seamlessly switch between HW
fixed-function blocks and shaders/CPU with no visible difference. Depending on
the content being displayed we might need to fallback to shaders/CPU at any
time. (A classic example would be a new notification popup preventing us from
leveraging KMS planes.)

> +An example of a drm_colorop object might look like one of these::
> +
> +    /* 1D enumerated curve */
> +    Color operation 42
> +    ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3 matrix, 3x4 matrix, 3D LUT, etc.} = 1D enumerated curve
> +    ├─ "BYPASS": bool {true, false}
> +    ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, sRGB inverse EOTF, PQ EOTF, PQ inverse EOTF, …}
> +    └─ "NEXT": immutable color operation ID = 43
> +
> +    /* custom 4k entry 1D LUT */
> +    Color operation 52
> +    ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3 matrix, 3x4 matrix, 3D LUT, etc.} = 1D LUT
> +    ├─ "BYPASS": bool {true, false}
> +    ├─ "SIZE": immutable range = 4096
> +    ├─ "DATA": blob
> +    └─ "NEXT": immutable color operation ID = 0
> +
> +    /* 17^3 3D LUT */
> +    Color operation 72
> +    ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3 matrix, 3x4 matrix, 3D LUT, etc.} = 3D LUT
> +    ├─ "BYPASS": bool {true, false}
> +    ├─ "3DLUT_MODES": read-only blob of supported 3DLUT modes
> +    ├─ "3DLUT_MODE_INDEX": index of selected 3DLUT mode

Nit: the 3D LUT modes have been dropped from the initial patch series.

> +    ├─ "DATA": blob
> +    └─ "NEXT": immutable color operation ID = 73

[...]

> +Color Pipeline Discovery
> +========================
> +
> +A DRM client wanting color management on a drm_plane will:
> +
> +1. Get the COLOR_PIPELINE property of the plane
> +2. iterate all COLOR_PIPELINE enum values
> +3. for each enum value walk the color pipeline (via the NEXT pointers)
> +   and see if the available color operations are suitable for the
> +   desired color management operations
> +
> +If userspace encounters an unknown or unsuitable color operation during
> +discovery it does not need to reject the entire color pipeline outright,
> +as long as the unknown or unsuitable colorop has a "BYPASS" property.
> +Drivers will ensure that a bypassed block does not have any effect.
> +
> +An example of chained properties to define an AMD pre-blending color
> +pipeline might look like this::
> +
> +    Plane 10
> +    ├─ "TYPE" (immutable) = Primary
> +    └─ "COLOR_PIPELINE": enum {0, 44} = 0
> +
> +    Color operation 44
> +    ├─ "TYPE" (immutable) = 1D enumerated curve
> +    ├─ "BYPASS": bool
> +    ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, PQ EOTF} = sRGB EOTF
> +    └─ "NEXT" (immutable) = 45
> +
> +    Color operation 45
> +    ├─ "TYPE" (immutable) = 3x4 Matrix
> +    ├─ "BYPASS": bool
> +    ├─ "DATA": blob
> +    └─ "NEXT" (immutable) = 46
> +
> +    Color operation 46
> +    ├─ "TYPE" (immutable) = 1D enumerated curve
> +    ├─ "BYPASS": bool
> +    ├─ "CURVE_1D_TYPE": enum {sRGB Inverse EOTF, PQ Inverse EOTF} = sRGB EOTF
> +    └─ "NEXT" (immutable) = 47
> +
> +    Color operation 47
> +    ├─ "TYPE" (immutable) = 1D LUT
> +    ├─ "SIZE": immutable range = 4096
> +    ├─ "DATA": blob
> +    └─ "NEXT" (immutable) = 48
> +
> +    Color operation 48
> +    ├─ "TYPE" (immutable) = 3D LUT
> +    ├─ "3DLUT_MODE_INDEX": 0

Ditto

> +    ├─ "DATA": blob
> +    └─ "NEXT" (immutable) = 49
> +
> +    Color operation 49
> +    ├─ "TYPE" (immutable) = 1D enumerated curve
> +    ├─ "BYPASS": bool
> +    ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, PQ EOTF} = sRGB EOTF
> +    └─ "NEXT" (immutable) = 0
> +
> +
> +Color Pipeline Programming
> +==========================
> +
> +Once a DRM client has found a suitable pipeline it will:
> +
> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
> +   drm_colorop object of the desired pipeline
> +2. Set the properties for all drm_colorop objects in the pipeline to the
> +   desired values, setting BYPASS to true for unused drm_colorop blocks,
> +   and false for enabled drm_colorop blocks
> +3. Perform (TEST_ONLY or not) atomic commit with all the other KMS
> +   states it wishes to change
> +
> +To configure the pipeline for an HDR10 PQ plane and blending in linear
> +space, a compositor might perform an atomic commit with the following
> +property values::
> +
> +    Plane 10
> +    └─ "COLOR_PIPELINE" = 42
> +
> +    Color operation 42
> +    └─ "BYPASS" = true
> +
> +    Color operation 44
> +    └─ "BYPASS" = true
> +
> +    Color operation 45
> +    └─ "BYPASS" = true
> +
> +    Color operation 46
> +    └─ "BYPASS" = true
> +
> +    Color operation 47
> +    ├─ "LUT_3D_DATA" = Gamut mapping + tone mapping + night mode

I think this is just named "DATA" now?

> +    └─ "BYPASS" = false
> +
> +    Color operation 48
> +    ├─ "CURVE_1D_TYPE" = PQ EOTF
> +    └─ "BYPASS" = false