[PATCH V8 03/43] drm/doc/rfc: Describe why prescriptive color pipeline is needed
Shengyu Qu
wiagn233 at outlook.com
Mon Mar 31 16:54:17 UTC 2025
You are right. Sorry for the noise.
在 2025/4/1 0:41, Alex Hung 写道:
>
>
> On 3/31/25 10:24, Shengyu Qu wrote:
>>
>>
>> 在 2025/3/27 7:46, Alex Hung 写道:
>>> From: Harry Wentland <harry.wentland at amd.com>
>>>
>>> Add documentation for color pipeline API.
>>>
>>> Signed-off-by: Alex Hung <alex.hung at amd.com>
>>> Signed-off-by: Harry Wentland <harry.wentland at amd.com>
>>> ---
>>> v8:
>>> - Fix typo "definint" -> "defining"
>>>
>>> v7:
>>> - Add a commit messages
>>>
>>> v5:
>>> - Don't require BYPASS to succeed (Sebastian)
>>> - use DATA for 1D and 3D LUT types (Sebastian)
>>> - update 3DLUT ops to use 3DLUT_MODES and 3DLUT_MODE_INDEX
>>> - Add section on drm_colorop extensibility
>>> - Add color_pipeline.rst to RFC toc tree
>>>
>>> v4:
>>> - Drop IOCTL docs since we dropped the IOCTLs (Pekka)
>>> - Clarify reading and setting of COLOR_PIPELINE prop (Pekka)
>>> - Add blurb about not requiring to reject a pipeline due to
>>> incompatible ops, as long as op can be bypassed (Pekka)
>>> - Dropped informational strings (such as input CSC) as they're
>>> not actually intended to be advertised (Pekka)
>>>
>>> v3:
>>> - Describe DRM_CLIENT_CAP_PLANE_COLOR_PIPELINE (Sebastian)
>>> - Ask for clear documentation of colorop behavior (Sebastian)
>>>
>>> v2:
>>> - Update colorop visualizations to match reality (Sebastian, Alex
>>> Hung)
>>> - Updated wording (Pekka)
>>> - Change BYPASS wording to make it non-mandatory (Sebastian)
>>> - Drop cover-letter-like paragraph from COLOR_PIPELINE Plane Property
>>> section (Pekka)
>>> - Use PQ EOTF instead of its inverse in Pipeline Programming
>>> example (Melissa)
>>> - Add "Driver Implementer's Guide" section (Pekka)
>>> - Add "Driver Forward/Backward Compatibility" section (Sebastian,
>>> Pekka)
>>>
>>> Documentation/gpu/rfc/color_pipeline.rst | 378 +++++++++++++++++++++++
>>> Documentation/gpu/rfc/index.rst | 3 +
>>> 2 files changed, 381 insertions(+)
>>> create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
>>>
>>> diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/
>>> Documentation/ gpu/rfc/color_pipeline.rst
>>> new file mode 100644
>>> index 000000000000..58bcc2a5ffd8
>>> --- /dev/null
>>> +++ b/Documentation/gpu/rfc/color_pipeline.rst
>>> @@ -0,0 +1,378 @@
>>> +.. SPDX-License-Identifier: GPL-2.0
>>> +
>>> +========================
>>> +Linux Color Pipeline API
>>> +========================
>>> +
>>> +What problem are we solving?
>>> +============================
>>> +
>>> +We would like to support pre-, and post-blending complex color
>>> +transformations in display controller hardware in order to allow for
>>> +HW-supported HDR use-cases, as well as to provide support to
>>> +color-managed applications, such as video or image editors.
>>> +
>>> +It is possible to support an HDR output on HW supporting the Colorspace
>>> +and HDR Metadata drm_connector properties, but that requires the
>>> +compositor or application to render and compose the content into one
>>> +final buffer intended for display. Doing so is costly.
>>> +
>>> +Most modern display HW offers various 1D LUTs, 3D LUTs, matrices,
>>> and other
>>> +operations to support color transformations. These operations are often
>>> +implemented in fixed-function HW and therefore much more power
>>> efficient than
>>> +performing similar operations via shaders or CPU.
>>> +
>>> +We would like to make use of this HW functionality to support
>>> complex color
>>> +transformations with no, or minimal CPU or shader load.
>>> +
>>> +
>>> +How are other OSes solving this problem?
>>> +========================================
>>> +
>>> +The most widely supported use-cases regard HDR content, whether
>>> video or
>>> +gaming.
>>> +
>>> +Most OSes will specify the source content format (color gamut,
>>> encoding transfer
>>> +function, and other metadata, such as max and average light levels)
>>> to a driver.
>>> +Drivers will then program their fixed-function HW accordingly to map
>>> from a
>>> +source content buffer's space to a display's space.
>>> +
>>> +When fixed-function HW is not available the compositor will assemble
>>> a shader to
>>> +ask the GPU to perform the transformation from the source content
>>> format to the
>>> +display's format.
>>> +
>>> +A compositor's mapping function and a driver's mapping function are
>>> usually
>>> +entirely separate concepts. On OSes where a HW vendor has no insight
>>> into
>>> +closed-source compositor code such a vendor will tune their color
>>> management
>>> +code to visually match the compositor's. On other OSes, where both
>>> mapping
>>> +functions are open to an implementer they will ensure both mappings
>>> match.
>>> +
>>> +This results in mapping algorithm lock-in, meaning that no-one alone
>>> can
>>> +experiment with or introduce new mapping algorithms and achieve
>>> +consistent results regardless of which implementation path is taken.
>>> +
>>> +Why is Linux different?
>>> +=======================
>>> +
>>> +Unlike other OSes, where there is one compositor for one or more
>>> drivers, on
>>> +Linux we have a many-to-many relationship. Many compositors; many
>>> drivers.
>>> +In addition each compositor vendor or community has their own view
>>> of how
>>> +color management should be done. This is what makes Linux so beautiful.
>>> +
>>> +This means that a HW vendor can now no longer tune their driver to one
>>> +compositor, as tuning it to one could make it look fairly different
>>> from
>>> +another compositor's color mapping.
>>> +
>>> +We need a better solution.
>>> +
>>> +
>>> +Descriptive API
>>> +===============
>>> +
>>> +An API that describes the source and destination colorspaces is a
>>> descriptive
>>> +API. It describes the input and output color spaces but does not
>>> describe
>>> +how precisely they should be mapped. Such a mapping includes many
>>> minute
>>> +design decision that can greatly affect the look of the final result.
>>> +
>>> +It is not feasible to describe such mapping with enough detail to
>>> ensure the
>>> +same result from each implementation. In fact, these mappings are a
>>> very active
>>> +research area.
>>> +
>>> +
>>> +Prescriptive API
>>> +================
>>> +
>>> +A prescriptive API describes not the source and destination
>>> colorspaces. It
>>> +instead prescribes a recipe for how to manipulate pixel values to
>>> arrive at the
>>> +desired outcome.
>>> +
>>> +This recipe is generally an ordered list of straight-forward
>>> operations,
>>> +with clear mathematical definitions, such as 1D LUTs, 3D LUTs,
>>> matrices,
>>> +or other operations that can be described in a precise manner.
>>> +
>>> +
>>> +The Color Pipeline API
>>> +======================
>>> +
>>> +HW color management pipelines can significantly differ between HW
>>> +vendors in terms of availability, ordering, and capabilities of HW
>>> +blocks. This makes a common definition of color management blocks and
>>> +their ordering nigh impossible. Instead we are defining an API that
>>> +allows user space to discover the HW capabilities in a generic manner,
>>> +agnostic of specific drivers and hardware.
>>> +
>>> +
>>> +drm_colorop Object
>>> +==================
>>> +
>>> +To support the definition of color pipelines we define the DRM core
>>> +object type drm_colorop. Individual drm_colorop objects will be chained
>>> +via the NEXT property of a drm_colorop to constitute a color pipeline.
>>> +Each drm_colorop object is unique, i.e., even if multiple color
>>> +pipelines have the same operation they won't share the same drm_colorop
>>> +object to describe that operation.
>>> +
>>> +Note that drivers are not expected to map drm_colorop objects
>>> statically
>>> +to specific HW blocks. The mapping of drm_colorop objects is entirely a
>>> +driver-internal detail and can be as dynamic or static as a driver
>>> needs
>>> +it to be. See more in the Driver Implementation Guide section below.
>>> +
>>> +Each drm_colorop has three core properties:
>>> +
>>> +TYPE: An enumeration property, defining the type of transformation,
>>> such as
>>> +* enumerated curve
>>> +* custom (uniform) 1D LUT
>>> +* 3x3 matrix
>>> +* 3x4 matrix
>>> +* 3D LUT
>>> +* etc.
>>> +
>>> +Depending on the type of transformation other properties will describe
>>> +more details.
>>> +
>>> +BYPASS: A boolean property that can be used to easily put a block into
>>> +bypass mode. The BYPASS property is not mandatory for a colorop, as
>>> long
>>> +as the entire pipeline can get bypassed by setting the
>>> COLOR_PIPELINE on
>>> +a plane to '0'.
>>> +
>>> +NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
>>> +drm_colorop is the last in the chain.
>>> +
>>> +An example of a drm_colorop object might look like one of these::
>>> +
>>> + /* 1D enumerated curve */
>>> + Color operation 42
>>> + ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3
>>> matrix, 3x4 matrix, 3D LUT, etc.} = 1D enumerated curve
>>> + ├─ "BYPASS": bool {true, false}
>>> + ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, sRGB inverse EOTF, PQ EOTF,
>>> PQ inverse EOTF, …}
>>> + └─ "NEXT": immutable color operation ID = 43
>>> +
>>> + /* custom 4k entry 1D LUT */
>>> + Color operation 52
>>> + ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3
>>> matrix, 3x4 matrix, 3D LUT, etc.} = 1D LUT
>>> + ├─ "BYPASS": bool {true, false}
>>> + ├─ "SIZE": immutable range = 4096
>>> + ├─ "DATA": blob
>>> + └─ "NEXT": immutable color operation ID = 0
>>> +
>>> + /* 17^3 3D LUT */
>>> + Color operation 72
>>> + ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3
>>> matrix, 3x4 matrix, 3D LUT, etc.} = 3D LUT
>>> + ├─ "BYPASS": bool {true, false}
>>> + ├─ "3DLUT_MODES": read-only blob of supported 3DLUT modes
>>> + ├─ "3DLUT_MODE_INDEX": index of selected 3DLUT mode
>>> + ├─ "DATA": blob
>>> + └─ "NEXT": immutable color operation ID = 73
>>> +
>>> +drm_colorop extensibility
>>> +-------------------------
>>> +
>>> +Unlike existing DRM core objects, like &drm_plane, drm_colorop is not
>>> +extensible. This simplifies implementations and keeps all functionality
>>> +for managing &drm_colorop objects in the DRM core.
>>> +
>>> +If there is a need one may introduce a simple &drm_colorop_funcs
>>> +function table in the future, for example to support an IN_FORMATS
>>> +property on a &drm_colorop.
>>> +
>>> +If a driver requires the ability to create a driver-specific colorop
>>> +object they will need to add &drm_colorop func table support with
>>> +support for the usual functions, like destroy, atomic_duplicate_state,
>>> +and atomic_destroy_state.
>> As far as I know, there are multiple embedded SOCs(with mainline linux
>> support) that support colorops which are not exist in current func
>> list. Why don't we just make the api extensible now? It would make the
>> API/ code more simple when adding colorops.
>
> Any vendor can create their color pipeline(s) and add new colorop
> objects accordingly. This patchset creates an AMD-specific color
> pipeline and colorop objects (for an obvious reason) as an demonstration
> and other vendors are working on their own too.
>
> The following paragraphs should answer your questions.
>
>>
>>> +
>>> +
>>> +COLOR_PIPELINE Plane Property
>>> +=============================
>>> +
>>> +Color Pipelines are created by a driver and advertised via a new
>>> +COLOR_PIPELINE enum property on each plane. Values of the property
>>> +always include object id 0, which is the default and means all color
>>> +processing is disabled. Additional values will be the object IDs of the
>>> +first drm_colorop in a pipeline. A driver can create and advertise
>>> none,
>>> +one, or more possible color pipelines. A DRM client will select a color
>>> +pipeline by setting the COLOR PIPELINE to the respective value.
>>> +
>>> +NOTE: Many DRM clients will set enumeration properties via the string
>>> +value, often hard-coding it. Since this enumeration is generated based
>>> +on the colorop object IDs it is important to perform the Color Pipeline
>>> +Discovery, described below, instead of hard-coding color pipeline
>>> +assignment. Drivers might generate the enum strings dynamically.
>>> +Hard-coded strings might only work for specific drivers on a specific
>>> +pieces of HW. Color Pipeline Discovery can work universally, as long as
>>> +drivers implement the required color operations.
>>> +
>>> +The COLOR_PIPELINE property is only exposed when the
>>> +DRM_CLIENT_CAP_PLANE_COLOR_PIPELINE is set. Drivers shall ignore any
>>> +existing pre-blend color operations when this cap is set, such as
>>> +COLOR_RANGE and COLOR_ENCODING. If drivers want to support COLOR_RANGE
>>> +or COLOR_ENCODING functionality when the color pipeline client cap is
>>> +set, they are expected to expose colorops in the pipeline to allow for
>>> +the appropriate color transformation.
>>> +
>>> +Setting of the COLOR_PIPELINE plane property or drm_colorop properties
>>> +is only allowed for userspace that sets this client cap.
>>> +
>>> +An example of a COLOR_PIPELINE property on a plane might look like
>>> this::
>>> +
>>> + Plane 10
>>> + ├─ "TYPE": immutable enum {Overlay, Primary, Cursor} = Primary
>>> + ├─ …
>>> + └─ "COLOR_PIPELINE": enum {0, 42, 52} = 0
>>> +
>>> +
>>> +Color Pipeline Discovery
>>> +========================
>>> +
>>> +A DRM client wanting color management on a drm_plane will:
>>> +
>>> +1. Get the COLOR_PIPELINE property of the plane
>>> +2. iterate all COLOR_PIPELINE enum values
>>> +3. for each enum value walk the color pipeline (via the NEXT pointers)
>>> + and see if the available color operations are suitable for the
>>> + desired color management operations
>>> +
>>> +If userspace encounters an unknown or unsuitable color operation during
>>> +discovery it does not need to reject the entire color pipeline
>>> outright,
>>> +as long as the unknown or unsuitable colorop has a "BYPASS" property.
>>> +Drivers will ensure that a bypassed block does not have any effect.
>>> +
>>> +An example of chained properties to define an AMD pre-blending color
>>> +pipeline might look like this::
>>> +
>>> + Plane 10
>>> + ├─ "TYPE" (immutable) = Primary
>>> + └─ "COLOR_PIPELINE": enum {0, 44} = 0
>>> +
>>> + Color operation 44
>>> + ├─ "TYPE" (immutable) = 1D enumerated curve
>>> + ├─ "BYPASS": bool
>>> + ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, PQ EOTF} = sRGB EOTF
>>> + └─ "NEXT" (immutable) = 45
>>> +
>>> + Color operation 45
>>> + ├─ "TYPE" (immutable) = 3x4 Matrix
>>> + ├─ "BYPASS": bool
>>> + ├─ "DATA": blob
>>> + └─ "NEXT" (immutable) = 46
>>> +
>>> + Color operation 46
>>> + ├─ "TYPE" (immutable) = 1D enumerated curve
>>> + ├─ "BYPASS": bool
>>> + ├─ "CURVE_1D_TYPE": enum {sRGB Inverse EOTF, PQ Inverse EOTF} =
>>> sRGB EOTF
>>> + └─ "NEXT" (immutable) = 47
>>> +
>>> + Color operation 47
>>> + ├─ "TYPE" (immutable) = 1D LUT
>>> + ├─ "SIZE": immutable range = 4096
>>> + ├─ "DATA": blob
>>> + └─ "NEXT" (immutable) = 48
>>> +
>>> + Color operation 48
>>> + ├─ "TYPE" (immutable) = 3D LUT
>>> + ├─ "3DLUT_MODE_INDEX": 0
>>> + ├─ "DATA": blob
>>> + └─ "NEXT" (immutable) = 49
>>> +
>>> + Color operation 49
>>> + ├─ "TYPE" (immutable) = 1D enumerated curve
>>> + ├─ "BYPASS": bool
>>> + ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, PQ EOTF} = sRGB EOTF
>>> + └─ "NEXT" (immutable) = 0
>>> +
>>> +
>>> +Color Pipeline Programming
>>> +==========================
>>> +
>>> +Once a DRM client has found a suitable pipeline it will:
>>> +
>>> +1. Set the COLOR_PIPELINE enum value to the one pointing at the first
>>> + drm_colorop object of the desired pipeline
>>> +2. Set the properties for all drm_colorop objects in the pipeline to
>>> the
>>> + desired values, setting BYPASS to true for unused drm_colorop
>>> blocks,
>>> + and false for enabled drm_colorop blocks
>>> +3. Perform (TEST_ONLY or not) atomic commit with all the other KMS
>>> + states it wishes to change
>>> +
>>> +To configure the pipeline for an HDR10 PQ plane and blending in linear
>>> +space, a compositor might perform an atomic commit with the following
>>> +property values::
>>> +
>>> + Plane 10
>>> + └─ "COLOR_PIPELINE" = 42
>>> +
>>> + Color operation 42
>>> + └─ "BYPASS" = true
>>> +
>>> + Color operation 44
>>> + └─ "BYPASS" = true
>>> +
>>> + Color operation 45
>>> + └─ "BYPASS" = true
>>> +
>>> + Color operation 46
>>> + └─ "BYPASS" = true
>>> +
>>> + Color operation 47
>>> + ├─ "LUT_3D_DATA" = Gamut mapping + tone mapping + night mode
>>> + └─ "BYPASS" = false
>>> +
>>> + Color operation 48
>>> + ├─ "CURVE_1D_TYPE" = PQ EOTF
>>> + └─ "BYPASS" = false
>>> +
>>> +
>>> +Driver Implementer's Guide
>>> +==========================
>>> +
>>> +What does this all mean for driver implementations? As noted above the
>>> +colorops can map to HW directly but don't need to do so. Here are some
>>> +suggestions on how to think about creating your color pipelines:
>>> +
>>> +- Try to expose pipelines that use already defined colorops, even if
>>> + your hardware pipeline is split differently. This allows existing
>>> + userspace to immediately take advantage of the hardware.
>>> +
>>> +- Additionally, try to expose your actual hardware blocks as colorops.
>>> + Define new colorop types where you believe it can offer significant
>>> + benefits if userspace learns to program them.
>>> +
>>> +- Avoid defining new colorops for compound operations with very narrow
>>> + scope. If you have a hardware block for a special operation that
>>> + cannot be split further, you can expose that as a new colorop type.
>>> + However, try to not define colorops for "use cases", especially if
>>> + they require you to combine multiple hardware blocks.
>>> +
>>> +- Design new colorops as prescriptive, not descriptive; by the
>>> + mathematical formula, not by the assumed input and output.
>>> +
>>> +A defined colorop type must be deterministic. The exact behavior of the
>>> +colorop must be documented entirely, whether via a mathematical formula
>>> +or some other description. Its operation can depend only on its
>>> +properties and input and nothing else, allowed error tolerance
>>> +notwithstanding.
>>> +
>>> +
>>> +Driver Forward/Backward Compatibility
>>> +=====================================
>>> +
>>> +As this is uAPI drivers can't regress color pipelines that have been
>>> +introduced for a given HW generation. New HW generations are free to
>>> +abandon color pipelines advertised for previous generations.
>>> +Nevertheless, it can be beneficial to carry support for existing color
>>> +pipelines forward as those will likely already have support in DRM
>>> +clients.
>>> +
>>> +Introducing new colorops to a pipeline is fine, as long as they can be
>>> +bypassed or are purely informational. DRM clients implementing support
>>> +for the pipeline can always skip unknown properties as long as they can
>>> +be confident that doing so will not cause unexpected results.
>>> +
>>> +If a new colorop doesn't fall into one of the above categories
>>> +(bypassable or informational) the modified pipeline would be unusable
>>> +for user space. In this case a new pipeline should be defined.
>>> +
>>> +
>>> +References
>>> +==========
>>> +
>>> +1. https://lore.kernel.org/dri-devel/
>>> QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn488=@emersion.fr/
>>> \ No newline at end of file
>>> diff --git a/Documentation/gpu/rfc/index.rst b/Documentation/gpu/rfc/
>>> index.rst
>>> index 396e535377fb..ef19b0ba2a3e 100644
>>> --- a/Documentation/gpu/rfc/index.rst
>>> +++ b/Documentation/gpu/rfc/index.rst
>>> @@ -35,3 +35,6 @@ host such documentation:
>>> .. toctree::
>>> i915_vm_bind.rst
>>> +
>>> +.. toctree::
>>> + color_pipeline.rst
>>> \ No newline at end of file
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xE3520CC91929C8E7.asc
Type: application/pgp-keys
Size: 6868 bytes
Desc: OpenPGP public key
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20250401/a1f3e4a1/attachment-0001.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20250401/a1f3e4a1/attachment-0001.sig>
More information about the wayland-devel
mailing list