HDR support in Wayland/Weston

Sharma, Shashank shashank.sharma at intel.com
Thu Jan 10 15:09:24 UTC 2019


If the block diagram is not aligned due to mail client, please refer to 
the attached .txt file. Hope thats slightly better :).

Regards

Shashank

On 1/10/2019 8:32 PM, Sharma, Shashank wrote:
> Hello All,
>
> This mail is to propose a design for enabling HDR support in 
> Wayland/Weston stack, using display engine capabilities, and get more 
> feedback and input from community.
> Here are few points (you might already know these), about HDR 
> framebuffers, videos and displays:
> - HDR content/buffers are composed in REC2020 colorspace, with bit 
> depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016.
> - HDR content come with their own Metadata to be applied to get the 
> right luminance at the display device.
>     - The metadata can be of two type 1. static 2. dynamic . For 
> simplicity, this solution is focusing on static HDR only (HDR10 standard)
> - HDR content also provide its supported EOTF (electro optical 
> transfer function) information, which is a curve (like SRGB gamma 
> curve). One popular EOTF is PQ(ST2084).
> - HDR capable displays mention their EOTF and HDR metadata support 
> information in EDID CEA-861-G blocks.
> - Normal SRGB buffers are composed in SRGB color space following 
> REC709 specifications.
> - For accurate blending in display engines, we need to make sure 
> following:
>     - All the buffers are in same colorspace (Rec 709 or Rec 2020)
>     - All the buffers are liner (gamma/EOTF removed)
>     - All the buffers are tone mapped in same zone (HDR or SDR)
>
> Please refer to the block diagram below, which presents a simple case 
> of a HDR P010 movie playback, with HDR buffers as video buffers, and 
> SDR buffers as subtitles. The subsystem looks and works like this:
> - A client decodes the buffer (using FFMpeg for example) and gets the 
> two buffers, one with video (HDR) and one subtitles (SDR)
> - Client passes following information to the compositor:
>      - The actual buffers
>      - Their colorspace infromation, BT2020 for HDR buffer, REC709 for 
> SDR buffer (planning to add a new protocol extension for this)
>      - The HDR metadata of the content (planning to add new protocol 
> for this)
>
> - Compositors actions:
>    - Reads the End display's HDR capabilities from display EDID. 
> Assume its an HDR HDMI monitor.
>    - Compositor tone maps every view's framebuffer to match tone of 
> end display, applying a libVA filter. In this example:
>         - The SDR subtitles frame will go through SDR to HDR tone 
> mapping (called S2H)
>         - The HDR video frame will go through HDR to HDR tone mapping 
> (called H2H) if the HDR capabilities of monitor and content are different.
>         - Now both the buffers and the monitor are in the same tone 
> mapped range.
>     - As the end display is HDR capable, and one of the content frame 
> is HDR, the compositor will prepare all other planes for color space 
> conversion (CSC) from REC709->REC2020 using plane CSC property.
>     - As the CSC and blending should be done in liner space, 
> compositor will also use plane level degamma to make the buffers linear.
>     - These actions will make sure that, during blending:
>         - All the buffers are in same colorspace (REC2020)
>         - All the buffers are linear
>         - All the buffers are tone mapped (HDR)
>         - The plane level color properties patch, for DRM can be found 
> here: https://patchwork.freedesktop.org/series/30875/
>     - Now, in order to re-apply the HDR curve, compositor will apply 
> CRTC level gamma, so that the output buffer is non-linear again.
>     - To pass the output HDR information to kernel, so that it can 
> create and send AVI-info-frames to HDMI, compositor will set Connector 
> HDR metadata property.
>         - Code for the same can be found here: 
> https://patchwork.freedesktop.org/series/25091/
>     - And they will ever live happily after :).
>
> Please provide inputs, feedbacks and suggestions for this design and 
> plan, so that we can improve out half cooked solution, and start 
> sending the patches.
>
>                      +------------------+ +-------------------+
>                      | SDR Buffer subtitles       | HDR Buffer video
>                      | (REC  709 colorsp)         | (REC 2020 colorsp |
>                      |                  | |                   |
>                      +-------+----------+ +-------+-----------+
>                              |                            |
>                              |                            |
>                              |                            |
> +------v---------------------------v------------+ +--------------+
>                       |   Compositor: v           |         | 
> LibVA        |
>                       |   - assigns views to overlays                 
> +---------> Tone mapping |
>                       |   - prepare plane/CRTC color properties       
> <---------+ SDR to HDR   |
>                       |     for linear blending in display            
> |         | HDR to SDR   |
> +------+-----------------------------+----------+ +--------------+
>                              |                             |
>                              | Tone mapped                 | Tone mapped
>                              | non-linear-Rec709           | 
> non-linear Rec2020
>                       +------v------+ +-------v--------+
>                        SRGB Degamma |              |EOTF as degamma |
>                       |(Plane)      | |(Plane)         |
>                       |             | |                |
>                       +------+------+ +-------+--------+
> Tone mapped linear Rec 709   |                             |
>                       +------v------+                      |  Tone mapped
>                       | CSC/CTM     |                      | 
> non-linear Rec2020
>                       | REC709->2020|                      |
>                       |             |                      |
>                       +------+------+                      |
>                              | Tone mapped linear          |
>                              | Rec 2020                    |
> +------v-----------------------------v---------+
>                       | Blender                            |
> |                                              |
> +--------------------+-------------------------+
>                                            | Tone mapped linear Rec2020
> +--------------------v-------------------------+  Tone mapped
>                       |           OETF(CRTC Gamma, post blending)    
> |  non-linear Rec2020 +------------------+
> | +---------------->    |  HDMI monitor    |
> +----------------------------------------------+ +------------------+
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20190110/b4cfe9ee/attachment-0001.html>
-------------- next part --------------
                     +------------------+         +-------------------+
                     | SDR Buffer subtitles       | HDR Buffer video
                     | (REC  709 colorsp)         | (REC 2020 colorsp |
                     |                  |         |                   |
                     +-------+----------+         +-------+-----------+
                             |                            |
                             |                            |
                             |                            |
                      +------v---------------------------v------------+         +--------------+
                      |   Compositor:                     v           |         | LibVA        |
                      |   - assigns views to overlays                 +---------> Tone mapping |
                      |   - prepare plane/CRTC color properties       <---------+ SDR to HDR   |
                      |     for linear blending in display            |         | HDR to SDR   |
                      +------+-----------------------------+----------+         +--------------+
                             |                             |
                             | Tone mapped                 | Tone mapped
                             | non-linear-Rec709           | non-linear Rec2020
                      +------v------+              +-------v--------+
                       SRGB Degamma |              |EOTF as degamma |
                      |(Plane)      |              |(Plane)         |
                      |             |              |                |
                      +------+------+              +-------+--------+
Tone mapped linear Rec 709   |                             |
                      +------v------+                      |  Tone mapped
                      | CSC/CTM     |                      |  non-linear Rec2020
                      | REC709->2020|                      |
                      |             |                      |
                      +------+------+                      |
                             | Tone mapped linear          |
                             | Rec 2020                    |
                      +------v-----------------------------v---------+
                      |           Blender                            |
                      |                                              |
                      +--------------------+-------------------------+
                                           | Tone mapped linear Rec2020
                      +--------------------v-------------------------+  Tone mapped
                      |           OETF(CRTC Gamma, post blending)    |  non-linear Rec2020 +------------------+
                      |                                              +---------------->    |  HDMI monitor    |
                      +----------------------------------------------+                     +------------------+


More information about the wayland-devel mailing list