[RFC PATCH v4 00/42] Color Pipeline API w/ VKMS

Joshua Ashton joshua at froggi.es
Thu Feb 29 15:31:29 UTC 2024



On 2/29/24 10:43, Daniel Vetter wrote:
> On Mon, Feb 26, 2024 at 04:10:14PM -0500, Harry Wentland wrote:
>> This is an RFC set for a color pipeline API, along with a sample
>> implementation in VKMS. All the key API bits are here. VKMS now
>> supports two named transfer function colorops and two matrix
>> colorops. We have IGT tests that check all four of these colorops
>> with a pixel-by-pixel comparison that checks that these colorops
>> do what we expect them to do with a +/- 1 8 bpc code point margin.
> 
> So vkms is definitely great to make sure the igts are generic enough and
> somewhat useful, but ... does steam run on vkms too? I think that would be
> a really good test to show that the api we have here is actually useful
> for compositors in a cross-driver way, and not just a neat idea that
> doesn't survive harsh reality.
> 
> And yes I realize that's probably going to be a bunch of work, but I feel
> like the color pipeline discussion has dragged around enough in
> hypotheticals and concerns that I think it would really help a lot.

I don't think we have ever tested Steam/Gamescope on vkms.

The last time I tried stuff there, there was all the problems with the 
ttm page table tail thing for virtio stuff that made using Steam games + 
Gamescope unfeasable because Vulkan + bindless, but I have heard those 
are solved now?

I will have to try it again at the weekend and see where it's at.
I am willing to place my bets that some part of the stack will fall over 
relating to modifiers somehow... =P

But yes, testing there would be good too, as we have the full Steam Deck 
OLED HDR color pipeline implemented in shader-based composition 
validated as being 1:1 on a suite of HDR and SDR test images.
(That *will* definitely rely on 3D LUTs being tetrahedrally interpolated 
though)

I'll have a look at this at the weekend and also see about getting a 
Gamescope branch that uses the new wip colorop stuff.

- Joshie 🐸✨

> 
> Thoughts?
> -Sima
> 
>>
>> The big new change with v4 is the addition of an amdgpu color
>> pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support
>> the following:
>>
>> 1. 1D Curve EOTF
>> 2. 3x4 CTM
>> 3. Multiplier
>> 4. 1D Curve Inverse EOTF
>> 5. 1D LUT
>> 6. 1D Curve EOTF
>> 7. 1D LUT
>>
>> The supported curves for the 1D Curve type are:
>> - sRGB EOTF and its inverse
>> - PQ EOTF, scaled to [0.0, 125.0] and its inverse
>> - BT.2020/BT.709 OETF and its inverse
>>
>> Note that the 1st and 5th colorops take the EOTF or Inverse
>> OETF while the 3rd colorop takes the Inverse EOTF or OETF.
>>
>> We are working on two more ops for amdgpu, the HDR multiplier
>> and the 3DLUT, which will give us this:
>>
>> 1. 1D Curve EOTF
>> 2. 3x4 CTM
>> 3. HDR Multiplier
>> 4. 1D Curve Inverse EOTF
>> 5. 1D LUT
>> 6. 3D LUT
>> 7. 1D Curve EOTF
>> 8. 1D LUT
>>
>> This, essentially mirrors the color pipeline used by gamescope
>> and presented by Melissa Wen, with the exception of the DEGAM
>> LUT, which is not currently used. See
>> [1] https://indico.freedesktop.org/event/4/contributions/186/attachments/138/218/xdc2023-TheRainbowTreasureMap-MelissaWen.pdf
>>
>> After this we'd like to also add the following ops:
>> - Scaler (Informational only)
>> - Color Encoding, to replace drm_plane's COLOR_ENCODING
>> - Color Range, to replace drm_plane's COLOR_RANGE
>>
>> This patchset is grouped as follows:
>>   - Patches 1-3: couple general patches/fixes
>>   - Patches 4-7: introduce kunit to VKMS
>>   - Patch 7: description of motivation and details behind the
>>              Color Pipeline API. If you're reading nothing else
>>              but are interested in the topic I highly recommend
>>              you take a look at this.
>>   - Patches 7-27: DRM core and VKMS changes for color pipeline API
>>   - Patches 28-40: DRM core and amdgpu changes for color pipeline API
>>
>> VKMS patches could still be improved in a few ways, though the
>> payoff might be limited and I would rather focus on other work
>> at the moment. The most obvious thing to improve would be to
>> eliminate the hard-coded LUTs for identity, and sRGB, and replace
>> them with fixed-point math instead.
>>
>> There are plenty of things that I would like to see here but
>> haven't had a chance to look at. These will (hopefully) be
>> addressed in future iterations, either in VKMS or amdgpu:
>>   - Clear documentation for each drm_colorop_type
>>   - Add custom LUT colorops to VKMS
>>   - Add pre-blending 3DLUT
>>   - How to support HW which can't bypass entire pipeline?
>>   - Add ability to create colorops that don't have BYPASS
>>   - Can we do a LOAD / COMMIT model for LUTs (and other properties)?
>>   - read-only scaling colorop which defines scaling taps and position
>>   - read-only color format colorop to define supported color formats
>>     for a pipeline
>>   - named matrices, for things like converting YUV to RGB
>>
>> IGT tests can be found at
>> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1
>>
>> IGT patches are also being sent to the igt-dev mailing list.
>>
>> If you prefer a gitlab MR for review you can find it at
>> https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5
>>
>> v4:
>>   - Add amdgpu color pipeline (WIP)
>>   - Don't block setting of deprecated properties, instead pass client cap
>>     to atomic check so drivers can ignore these props
>>   - Drop IOCTL definitions (Pekka)
>>   - Use enum property for colorop TYPE (Pekka)
>>   - A few cleanups to the docs (Pekka)
>>   - Rework the TYPE enum to name relation to avoid code duplication (Pekka)
>>   - Add missing function declarations (Chaitanya Kumar Borah)
>>   - Allow setting of NEXT property to NULL in _set_ function (Chaitanya Kumar Borah)
>>   - Add helper for creation of pipeline drm_plane property (Pekka)
>>   - Always create Bypass pipeline (Pekka)
>>   - A bunch of changes to VKMS kunit tests (Pekka)
>>   - Fix index in CTM doc (Pekka)
>>
>> v3:
>>   - Abandon IOCTLs and discover colorops as clients iterate the pipeline
>>   - Remove need for libdrm
>>   - Add color_pipeline client cap and make mutually exclusive with
>>     COLOR_RANGE and COLOR_ENCODING properties
>>   - add CTM colorop to VKMS
>>   - Use include way for kunit testing static functions (Arthur)
>>   - Make TYPE a range property
>>   - Move enum drm_colorop_type to uapi header
>>   - and a bunch of smaller bits that are highlighted in the relevant commit
>>     description
>>
>> v2:
>>   - Rebased on drm-misc-next
>>   - Introduce a VKMS Kunit so we can test LUT functionality in vkms_composer
>>   - Incorporate feedback in color_pipeline.rst doc
>>   - Add support for sRGB inverse EOTF
>>   - Add 2nd enumerated TF colorop to VKMS
>>   - Fix LUTs and some issues with applying LUTs in VKMS
>>
>> Cc: Ville Syrjala <ville.syrjala at linux.intel.com>
>> Cc: Pekka Paalanen <pekka.paalanen at collabora.com>
>> Cc: Simon Ser <contact at emersion.fr>
>> Cc: Harry Wentland <harry.wentland at amd.com>
>> Cc: Melissa Wen <mwen at igalia.com>
>> Cc: Jonas Ådahl <jadahl at redhat.com>
>> Cc: Sebastian Wick <sebastian.wick at redhat.com>
>> Cc: Shashank Sharma <shashank.sharma at amd.com>
>> Cc: Alexander Goins <agoins at nvidia.com>
>> Cc: Joshua Ashton <joshua at froggi.es>
>> Cc: Michel Dänzer <mdaenzer at redhat.com>
>> Cc: Aleix Pol <aleixpol at kde.org>
>> Cc: Xaver Hugl <xaver.hugl at gmail.com>
>> Cc: Victoria Brekenfeld <victoria at system76.com>
>> Cc: Sima <daniel at ffwll.ch>
>> Cc: Uma Shankar <uma.shankar at intel.com>
>> Cc: Naseer Ahmed <quic_naseer at quicinc.com>
>> Cc: Christopher Braga <quic_cbraga at quicinc.com>
>> Cc: Abhinav Kumar <quic_abhinavk at quicinc.com>
>> Cc: Arthur Grillo <arthurgrillo at riseup.net>
>> Cc: Hector Martin <marcan at marcan.st>
>> Cc: Liviu Dudau <Liviu.Dudau at arm.com>
>> Cc: Sasha McIntosh <sashamcintosh at google.com>
>> Cc: Chaitanya Kumar Borah <chaitanya.kumar.borah at intel.com>
>>
>> Alex Hung (10):
>>    drm/colorop: define a new macro for_each_new_colorop_in_state
>>    drm/amd/display: Skip color pipeline initialization for cursor plane
>>    drm/amd/display: Add support for sRGB EOTF in DEGAM block
>>    drm/amd/display: Add support for sRGB Inverse EOTF in SHAPER block
>>    drm/amd/display: Add support for sRGB EOTF in BLND block
>>    drm/colorop: Add 1D Curve Custom LUT type
>>    drm/amd/display: add shaper and blend colorops for 1D Curve Custom LUT
>>    drm/amd/display: add 3x4 matrix colorop
>>    drm/colorop: Add mutliplier type
>>    drm/amd/display: add multiplier colorop
>>
>> Harry Wentland (32):
>>    drm: Don't treat 0 as -1 in drm_fixp2int_ceil
>>    drm: Add helper for conversion from signed-magnitude
>>    drm: Correctly round for fixp2int_round
>>    drm/vkms: Round fixp2int conversion in lerp_u16
>>    drm/vkms: Create separate Kconfig file for VKMS
>>    drm/vkms: Add kunit tests for VKMS LUT handling
>>    drm/vkms: Avoid reading beyond LUT array
>>    drm/doc/rfc: Describe why prescriptive color pipeline is needed
>>    drm/colorop: Introduce new drm_colorop mode object
>>    drm/colorop: Add TYPE property
>>    drm/colorop: Add 1D Curve subtype
>>    drm/colorop: Add BYPASS property
>>    drm/colorop: Add NEXT property
>>    drm/colorop: Add atomic state print for drm_colorop
>>    drm/plane: Add COLOR PIPELINE property
>>    drm/colorop: Add NEXT to colorop state print
>>    drm/vkms: Add enumerated 1D curve colorop
>>    drm/vkms: Add kunit tests for linear and sRGB LUTs
>>    drm/colorop: Introduce DRM_CLIENT_CAP_PLANE_COLOR_PIPELINE
>>    drm/colorop: Add 3x4 CTM type
>>    drm/vkms: Pull apply_colorop out of pre_blend_color_transform
>>    drm/vkms: Use s32 for internal color pipeline precision
>>    drm/vkms: add 3x4 matrix in color pipeline
>>    drm/tests: Add a few tests around drm_fixed.h
>>    drm/vkms: Add tests for CTM handling
>>    drm/colorop: pass plane_color_pipeline client cap to atomic check
>>    drm/amd/display: Ignore deprecated props when plane_color_pipeline set
>>    drm/amd/display: Add bypass COLOR PIPELINE
>>    drm/colorop: Add PQ 125 EOTF and its inverse
>>    drm/amd/display: Enable support for PQ 125 EOTF and Inverse
>>    drm/colorop: add BT2020/BT709 OETF and Inverse OETF
>>    drm/amd/display: Add support for BT.709 and BT.2020 TFs
>>
>>   Documentation/gpu/rfc/color_pipeline.rst      | 360 ++++++++
>>   drivers/gpu/drm/Kconfig                       |  14 +-
>>   drivers/gpu/drm/Makefile                      |   1 +
>>   .../gpu/drm/amd/display/amdgpu_dm/Makefile    |   3 +-
>>   .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c |   4 +
>>   .../amd/display/amdgpu_dm/amdgpu_dm_color.c   | 352 ++++++++
>>   .../amd/display/amdgpu_dm/amdgpu_dm_colorop.c | 160 ++++
>>   .../amd/display/amdgpu_dm/amdgpu_dm_colorop.h |  36 +
>>   .../amd/display/amdgpu_dm/amdgpu_dm_plane.c   |  32 +
>>   drivers/gpu/drm/drm_atomic.c                  | 161 +++-
>>   drivers/gpu/drm/drm_atomic_helper.c           |  12 +
>>   drivers/gpu/drm/drm_atomic_state_helper.c     |   5 +
>>   drivers/gpu/drm/drm_atomic_uapi.c             | 158 ++++
>>   drivers/gpu/drm/drm_colorop.c                 | 411 +++++++++
>>   drivers/gpu/drm/drm_ioctl.c                   |   7 +
>>   drivers/gpu/drm/drm_mode_config.c             |   7 +
>>   drivers/gpu/drm/drm_plane.c                   |  52 ++
>>   drivers/gpu/drm/tests/Makefile                |   3 +-
>>   drivers/gpu/drm/tests/drm_fixp_test.c         |  69 ++
>>   drivers/gpu/drm/vkms/Kconfig                  |  20 +
>>   drivers/gpu/drm/vkms/Makefile                 |   4 +-
>>   drivers/gpu/drm/vkms/tests/.kunitconfig       |   4 +
>>   drivers/gpu/drm/vkms/tests/vkms_color_tests.c | 449 ++++++++++
>>   drivers/gpu/drm/vkms/vkms_colorop.c           | 100 +++
>>   drivers/gpu/drm/vkms/vkms_composer.c          | 135 ++-
>>   drivers/gpu/drm/vkms/vkms_drv.h               |   8 +
>>   drivers/gpu/drm/vkms/vkms_luts.c              | 802 ++++++++++++++++++
>>   drivers/gpu/drm/vkms/vkms_luts.h              |  12 +
>>   drivers/gpu/drm/vkms/vkms_plane.c             |   2 +
>>   include/drm/drm_atomic.h                      | 122 +++
>>   include/drm/drm_atomic_uapi.h                 |   3 +
>>   include/drm/drm_colorop.h                     | 301 +++++++
>>   include/drm/drm_file.h                        |   7 +
>>   include/drm/drm_fixed.h                       |  35 +-
>>   include/drm/drm_mode_config.h                 |  18 +
>>   include/drm/drm_plane.h                       |  13 +
>>   include/uapi/drm/drm.h                        |  16 +
>>   include/uapi/drm/drm_mode.h                   |  14 +
>>   38 files changed, 3882 insertions(+), 30 deletions(-)
>>   create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
>>   create mode 100644 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_colorop.c
>>   create mode 100644 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_colorop.h
>>   create mode 100644 drivers/gpu/drm/drm_colorop.c
>>   create mode 100644 drivers/gpu/drm/tests/drm_fixp_test.c
>>   create mode 100644 drivers/gpu/drm/vkms/Kconfig
>>   create mode 100644 drivers/gpu/drm/vkms/tests/.kunitconfig
>>   create mode 100644 drivers/gpu/drm/vkms/tests/vkms_color_tests.c
>>   create mode 100644 drivers/gpu/drm/vkms/vkms_colorop.c
>>   create mode 100644 drivers/gpu/drm/vkms/vkms_luts.c
>>   create mode 100644 drivers/gpu/drm/vkms/vkms_luts.h
>>   create mode 100644 include/drm/drm_colorop.h
>>
>> --
>> 2.44.0
>>
> 



More information about the wayland-devel mailing list