[PATCH] unstable: add HDR Mastering Meta-data Protocol

[Graeme Gill]

Sharma, Shashank wrote:

>> From: Graeme Gill [mailto:graeme2 at argyllcms.com]

>>>> From: Ankit Nautiyal <ankit.k.nautiyal at intel.com>
>>>> 
>>>> This protcol enables a client to send the hdr meta-data: MAX-CLL, MAX-FALL, Max
>>>> Luminance and Min Luminance as defined by SMPTE ST.2086.
>> 
>> Hmm. I'm wondering what the intent of this is.

> The main reason is to pass it to the compositor so that it can do the tone mapping
> properly. As you know, there could be cases where we are dealing with one HDR and
> multiple SDR frames. Now, the HDR content comes with its own metadata information,
> which needs to be passed to the display using the AVI infoframes, for the correct HDR
> experience, but we need to make others non-HDR frames also compatible to this
> brightness range of metadata, so we do tone mapping.

I've been thinking a bit about that, and currently my view is
that this is not the best way of arranging things. MAX-CLL and MAX-FALL
are specific to certain current source of HD imagery, and not all HDR
sources will have those specific numbers, and video standards may change
in the future (i.e. frame by frame meta-data). Exactly how they can be
used is unclear - i.e. they are observations from a video stream, not
parameters for a specific tone mapping. So punting them to the Compositor
doesn't seem such a good approach.

Instead what I'd suggest is providing some concrete tone mapping
operators for HDR handling (at least for V1), with well defined behavior
and parameters. The video playback application can do the conversion
from source format specific observations like MAX-CLL and MAX-FALL
into the specific compositor tone mapping parameters, or it has the option of
determining those parameters some other way (user setting perhaps,
or it's own analysis of the video stream), or it can
even do the adaptation of the source to the display itself,
since it has access to the display characteristics via the ICC profile.

I think it's quite possible to have the Compositor do sane HDR conversions
with no other extra information than the HDR nominal diffuse white
brightness for each HDR profile, and perhaps this could be a base Compositor
tone mapping behavior, with other tone mapping algorithms (and
their associated parameters) added as they are identified and
defined or standardized.

>> If the idea is that somehow the compositor is to do dynamic HDR rendering, then I 
>> have my doubts about its suitability and practicality, unless you are prepared to
>> spell out in detail the algorithm it should use for performing this.
>> 
> The GL tone mapping algorithms will be opensource methods, and will be available for
> the code review. HW tone mapping is using the libVA interface and media-driver handler.

Right, but there is a distinction between a particular implementation and
a Compositor specification.

Can you write a specification of how the tone curves you are proposing
to use work, in such a way that someone else can implement them, and
that someone dealing with other HDR sources can use them ?
(i.e. HDR photographs or synthetic imagery from games etc.)

> Right now we are not targeting this, but eventually we want to be there. As we all
> know, it's a huge area to cover in an single attempt, and gets very cumbersome. So the
> plan is to first implement a modular limited capability HDR video playback, and  use
> that as a driving and testing vehicle for color management, and then later enhance it
> with adding more and more modules.

Sure. I'm a little worried though, that video specific HDR considerations will
end up getting set in stone, and hamper more general HDR compatibility.

Cheers,

Graeme Gill.