<html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> Hello All, This mail is to propose a design for enabling HDR support in Wayland/Weston stack, using display engine capabilities, and get more feedback and input from community. Here are few points (you might already know these), about HDR framebuffers, videos and displays: - HDR content/buffers are composed in REC2020 colorspace, with bit depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016. - HDR content come with their own Metadata to be applied to get the right luminance at the display device. - The metadata can be of two type 1. static 2. dynamic . For simplicity, this solution is focusing on static HDR only (HDR10 standard) - HDR content also provide its supported EOTF (electro optical transfer function) information, which is a curve (like SRGB gamma curve). One popular EOTF is PQ(ST2084). - HDR capable displays mention their EOTF and HDR metadata support information in EDID CEA-861-G blocks. <span style="color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: xx-large; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 100; letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"> - Normal SRGB buffers are composed in SRGB color space following REC709 specifications. - For accurate blending in display engines, we need to make sure following: - All the buffers are in same colorspace (Rec 709 or Rec 2020) - All the buffers are liner (gamma/EOTF removed) - All the buffers are tone mapped in same zone (HDR or SDR) Please refer to the block diagram below, which presents a simple case of a HDR P010 movie playback, with HDR buffers as video buffers, and SDR buffers as subtitles. The subsystem looks and works like this: - A client decodes the buffer (using FFMpeg for example) and gets the two buffers, one with video (HDR) and one subtitles (SDR) - Client passes following information to the compositor: - The actual buffers - Their colorspace infromation, BT2020 for HDR buffer, REC709 for SDR buffer (planning to add a new protocol extension for this) - The HDR metadata of the content (planning to add new protocol for this) - Compositors actions: - Reads the End display's HDR capabilities from display EDID. Assume its an HDR HDMI monitor. - Compositor tone maps every view's framebuffer to match tone of end display, applying a libVA filter. In this example: - The SDR subtitles frame will go through SDR to HDR tone mapping (called S2H) - The HDR video frame will go through HDR to HDR tone mapping (called H2H) if the HDR capabilities of monitor and content are different. - Now both the buffers and the monitor are in the same tone mapped range. - As the end display is HDR capable, and one of the content frame is HDR, the compositor will prepare all other planes for color space conversion (CSC) from REC709->REC2020 using plane CSC property. - As the CSC and blending should be done in liner space, compositor will also use plane level degamma to make the buffers linear. - These actions will make sure that, during blending: - All the buffers are in same colorspace (REC2020) - All the buffers are linear - All the buffers are tone mapped (HDR) - The plane level color properties patch, for DRM can be found here: <a class="moz-txt-link-freetext" href="https://patchwork.freedesktop.org/series/30875/">https://patchwork.freedesktop.org/series/30875/</a> - Now, in order to re-apply the HDR curve, compositor will apply CRTC level gamma, so that the output buffer is non-linear again. - To pass the output HDR information to kernel, so that it can create and send AVI-info-frames to HDMI, compositor will set Connector HDR metadata property. - Code for the same can be found here: <a class="moz-txt-link-freetext" href="https://patchwork.freedesktop.org/series/25091/">https://patchwork.freedesktop.org/series/25091/</a> - And they will ever live happily after :). Please provide inputs, feedbacks and suggestions for this design and plan, so that we can improve out half cooked solution, and start sending the patches. +------------------+ +-------------------+ | SDR Buffer subtitles | HDR Buffer video | (REC 709 colorsp) | (REC 2020 colorsp | | | | | +-------+----------+ +-------+-----------+ | | | | | | +------v---------------------------v------------+ +--------------+ | Compositor: v | | LibVA | | - assigns views to overlays +---------> Tone mapping | | - prepare plane/CRTC color properties <---------+ SDR to HDR | | for linear blending in display | | HDR to SDR | +------+-----------------------------+----------+ +--------------+ | | | Tone mapped | Tone mapped | non-linear-Rec709 | non-linear Rec2020 +------v------+ +-------v--------+ SRGB Degamma | |EOTF as degamma | |(Plane) | |(Plane) | | | | | +------+------+ +-------+--------+ Tone mapped linear Rec 709 | | +------v------+ | Tone mapped | CSC/CTM | | non-linear Rec2020 | REC709->2020| | | | | +------+------+ | | Tone mapped linear | | Rec 2020 | +------v-----------------------------v---------+ | Blender | | | +--------------------+-------------------------+ | Tone mapped linear Rec2020 +--------------------v-------------------------+ Tone mapped | OETF(CRTC Gamma, post blending) | non-linear Rec2020 +------------------+ | +----------------> | HDMI monitor | +----------------------------------------------+ +------------------+ </body> </html>