[PATCH] unstable: add HDR Mastering Meta-data Protocol

[Sharma, Shashank]

Regards
Shashank

> -----Original Message-----
> From: Graeme Gill [mailto:graeme2 at argyllcms.com]
> Sent: Tuesday, March 5, 2019 9:35 AM
> To: Pekka Paalanen <ppaalanen at gmail.com>; Nautiyal, Ankit K
> <ankit.k.nautiyal at intel.com>
> Cc: e.burema at gmail.com; niels_ole at salscheider-online.de; wayland-
> devel at lists.freedesktop.org; sebastian at sebastianwick.net; Kps, Harish Krupo
> <harish.krupo.kps at intel.com>; Sharma, Shashank <shashank.sharma at intel.com>
> Subject: Re: [PATCH] unstable: add HDR Mastering Meta-data Protocol
> 
> Pekka Paalanen wrote:
> > On Wed, 27 Feb 2019 10:27:16 +0530
> > "Nautiyal, Ankit K" <ankit.k.nautiyal at intel.com> wrote:
> >
> >> From: Ankit Nautiyal <ankit.k.nautiyal at intel.com>
> >>
> >> This protcol enables a client to send the hdr meta-data:
> >> MAX-CLL, MAX-FALL, Max Luminance and Min Luminance as defined by
> >> SMPTE ST.2086.
> 
> Hmm. I'm wondering what the intent of this is.
> 
> If it is to feed these values through to the display itself so that it can do dynamic HDR
> rendering, then this tends to cut across color management, since color management
> as it is currently formulated depends on reproducability in color devices. If really
> desired, I guess this would have to be some kind of special mode that bypasses the
> usual rendering pipeline.
> 
The main reason is to pass it to the compositor so that it can do the tone mapping properly. As you know, there could be cases where we are dealing with one HDR and multiple SDR frames. Now, the HDR content comes with its own metadata information, which needs to be passed to the display using the AVI infoframes, for the correct HDR experience, but we need to make others non-HDR frames also compatible to this brightness range of metadata, so we do tone mapping. 

We are writing compositor code, which will take a call on target outputs content brightness level too (apart with colorspace) by applying appropriate tone mapping (SDR to HDR, HDR to HDR, or HDR to SDR)  using libVA or GL extensions. This will make sure that everything being shown on the display falls into same brightness range. 
 
> If the idea is that somehow the compositor is to do dynamic HDR rendering, then I
> have my doubts about its suitability and practicality, unless you are prepared to spell
> out in detail the algorithm it should use for performing this.
> 
The GL tone mapping algorithms will be opensource methods, and will be available for the code review. HW tone mapping is using the libVA interface and media-driver handler.  
> My assumption was that the compositor would only be expected to do well
> understood fallback color conversions, and that anything that needed special
> functionality or that was going to do experimental color handling (such as dynamic
> HDR rendering) would do so in the client.
> 
Yes, we are trying our best here to make compositor fair and intelligent, and I believe with proper discussions and code reviews we should be able to reach that level. 
> For the purposes of mixing SDR and HDR in the compositor as a fallback, a much
> simpler set of transforms should be sufficient, namely a brightness for converting
> SDR into HDR, and a tone mapping curve or equivalent settings for compressing HDR
> into SDR.
> 
Agree. 
> > Is there a case for compositors that implement color management but
> > not HDR support? Would it be unreasonable to make HDR parameters a
> > part of the color management extension? Such that it would be optional
> > for a client to set the HDR parameters.
> 
Right now we are not targeting this, but eventually we want to be there. As we all know, it's a huge area to cover in an single attempt, and gets very cumbersome. So the plan is to first implement a modular limited capability HDR video playback, and  use that as a driving and testing vehicle for color management, and then later enhance it with adding more and more modules.  
> I think that at this point in time it is entirely reasonable that a compositing system have
> some level of HDR awareness.
> 
Yes, the idea is to make it completely aware of HDR scenarios over the period, over a slowly maturing stack.  

- Shashank
> > Btw. do HDR videos contain ICC profiles? Where would a client get the
> > ICC profile for a video it wants to show? Do we need to require the
> > compositor to expose some commonly used specific profiles?
> 
> As I understand it, no they don't. They either rely on a stated encoding (i.e. REC2020
> based), or sometimes have simple meta-data such as primary coordinates & transfer
> curve specifications. A client wanting to support such meta-data can simply create a
> matching ICC profile on the fly if it wishes.
> 
> Cheers,
> 	Graeme Gill.