HDR support in Wayland/Weston

[Sharma, Shashank]

Hello All,

This mail is to propose a design for enabling HDR support in 
Wayland/Weston stack, using display engine capabilities, and get more 
feedback and input from community.
Here are few points (you might already know these), about HDR 
framebuffers, videos and displays:
- HDR content/buffers are composed in REC2020 colorspace, with bit depth 
10/12/16 BPC. Some of the popular formats are P010,P012,P016.
- HDR content come with their own Metadata to be applied to get the 
right luminance at the display device.
     - The metadata can be of two type 1. static 2. dynamic . For 
simplicity, this solution is focusing on static HDR only (HDR10 standard)
- HDR content also provide its supported EOTF (electro optical transfer 
function) information, which is a curve (like SRGB gamma curve). One 
popular EOTF is PQ(ST2084).
- HDR capable displays mention their EOTF and HDR metadata support 
information in EDID CEA-861-G blocks.
- Normal SRGB buffers are composed in SRGB color space following REC709 
specifications.
- For accurate blending in display engines, we need to make sure following:
     - All the buffers are in same colorspace (Rec 709 or Rec 2020)
     - All the buffers are liner (gamma/EOTF removed)
     - All the buffers are tone mapped in same zone (HDR or SDR)

Please refer to the block diagram below, which presents a simple case of 
a HDR P010 movie playback, with HDR buffers as video buffers, and SDR 
buffers as subtitles. The subsystem looks and works like this:
- A client decodes the buffer (using FFMpeg for example) and gets the 
two buffers, one with video (HDR) and one subtitles (SDR)
- Client passes following information to the compositor:
      - The actual buffers
      - Their colorspace infromation, BT2020 for HDR buffer, REC709 for 
SDR buffer (planning to add a new protocol extension for this)
      - The HDR metadata of the content (planning to add new protocol 
for this)

- Compositors actions:
    - Reads the End display's HDR capabilities from display EDID. Assume 
its an HDR HDMI monitor.
    - Compositor tone maps every view's framebuffer to match tone of end 
display, applying a libVA filter. In this example:
         - The SDR subtitles frame will go through SDR to HDR tone 
mapping (called S2H)
         - The HDR video frame will go through HDR to HDR tone mapping 
(called H2H) if the HDR capabilities of monitor and content are different.
         - Now both the buffers and the monitor are in the same tone 
mapped range.
     - As the end display is HDR capable, and one of the content frame 
is HDR, the compositor will prepare all other planes for color space 
conversion (CSC) from REC709->REC2020 using plane CSC property.
     - As the CSC and blending should be done in liner space, compositor 
will also use plane level degamma to make the buffers linear.
     - These actions will make sure that, during blending:
         - All the buffers are in same colorspace (REC2020)
         - All the buffers are linear
         - All the buffers are tone mapped (HDR)
         - The plane level color properties patch, for DRM can be found 
here: https://patchwork.freedesktop.org/series/30875/
     - Now, in order to re-apply the HDR curve, compositor will apply 
CRTC level gamma, so that the output buffer is non-linear again.
     - To pass the output HDR information to kernel, so that it can 
create and send AVI-info-frames to HDMI, compositor will set Connector 
HDR metadata property.
         - Code for the same can be found here: 
https://patchwork.freedesktop.org/series/25091/
     - And they will ever live happily after :).

Please provide inputs, feedbacks and suggestions for this design and 
plan, so that we can improve out half cooked solution, and start sending 
the patches.

                      +------------------+ +-------------------+
                      | SDR Buffer subtitles       | HDR Buffer video
                      | (REC  709 colorsp)         | (REC 2020 colorsp |
                      |                  | |                   |
                      +-------+----------+ +-------+-----------+
                              |                            |
                              |                            |
                              |                            |
+------v---------------------------v------------+ +--------------+
                       |   Compositor: v           |         | 
LibVA        |
                       |   - assigns views to overlays                 
+---------> Tone mapping |
                       |   - prepare plane/CRTC color properties       
<---------+ SDR to HDR   |
                       |     for linear blending in display            
|         | HDR to SDR   |
+------+-----------------------------+----------+ +--------------+
                              |                             |
                              | Tone mapped                 | Tone mapped
                              | non-linear-Rec709           | non-linear 
Rec2020
                       +------v------+ +-------v--------+
                        SRGB Degamma |              |EOTF as degamma |
                       |(Plane)      |              |(Plane) |
                       |             |              | |
                       +------+------+ +-------+--------+
Tone mapped linear Rec 709   |                             |
                       +------v------+                      |  Tone mapped
                       | CSC/CTM     |                      | non-linear 
Rec2020
                       | REC709->2020|                      |
                       |             |                      |
                       +------+------+                      |
                              | Tone mapped linear          |
                              | Rec 2020                    |
+------v-----------------------------v---------+
                       |           Blender |
                       | |
+--------------------+-------------------------+
                                            | Tone mapped linear Rec2020
+--------------------v-------------------------+  Tone mapped
                       |           OETF(CRTC Gamma, post blending) |  
non-linear Rec2020 +------------------+
                       | +---------------->    |  HDMI monitor    |
+----------------------------------------------+ +------------------+



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20190110/e60da872/attachment.html>