HDR support in Wayland/Weston

[Graeme Gill]

Pekka Paalanen wrote:

Hi Pekka,
	thanks for your response.

>> As far as I was informed, Wayland
>> is architected in such a way that this is not possible, since clients
>> have no knowledge of which display the pixels they send will end up on.
> 
> Nothing has changed there.

I've been pondering the various Color Management (CM) approaches to
working around this limitation, but I keep coming back to it
as the most fruitful direction to talk about. The main reasons
are that this implies the least extra burden on Wayland implementations,
and is most consonant with current application and GUI toolkit
CM code.

And in fact Wayland has already been changed in this direction already, to
accommodate a highly analogous requirement to that of Color Management :- HiDPI.
So as best I understand it, because (unlike X11) Wayland does not do
rendering, it is not reasonable for it to suddenly re-render application
graphics at hi resolution - the best that it can do is scale the pixels,
leading to poorer visual quality than is possible on HiDPI displays.
So HiDPI aware applications have to know when they are rendering for a HiDPI
output, and scale their buffers and rendering accordingly, and tell Wayland that
they have done so using wl_surface.set_buffer_scale. [ I am not currently
clear on how the situation of a user window straddling two displays of
differing DPI is handled. ]

The CM situation is highly analogous - like the DPI, the colorspace (profile)
of each output may be different for different displays, so for highest
quality output, something in the graphics chain needs to accommodate it.
To do so the relevant information is needed :- what output is being
rendered to, and what its characteristic is (DPI / Color Profile).

If done in the composer, it would need to render the graphic elements to
the output DPI / convert the source colorspace to the output colorspace.
But the composer would need the code to do rendering / convert colorspaces
(as well as being told what the graphic elements / source colorspace is),
and this is not the role Wayland has - that's the responsibility of the
client, so instead Wayland makes it possible for the client to know what DPI
it is rendering to. The analogous facility for CM is for the client to know
what output colorspace it is rendering for.

The clean/simplest approach to HDR is to treat it as just another output
colorspace, where it is up to the application to render the color
it intends to display, and the Wayland compositor is compositing everything
for that output in that HDR colorspace.

Now practically speaking this would assume that all Wayland clients connected
to an HDR display in HDR mode are CM/HDR aware, which is rather unlikely. So
some backward compatibility modes might be highly desirable (I have
some thoughts on that), but in any case, it would also help the quality
of such backward compatibility _and_ compositing (i.e. linear light
compositing option), if Wayland at least had access to the output color
profiles. So there is a lot of advantage in Wayland providing the
registry/API of output color profiles both for itself, and clients.

> Wayland and apps need to provide the compositor all the necessary
> information for the compositor to do all the conversions, mapping and
> blending correctly, if it has to.

And perhaps it shouldn't have to.

> This is because an application will provide only one image for a
> wl_surface, and the compositor may show that on any number of any kind
> of outputs at once.

That's a big problem. That assumes either that all displays are interchangeable
at the pixel level (they are not - they have different DPI and Colorspaces/Gamut/HDR
capability), or that Wayland has to know how to re-render to accommodate those
difference (and Wayland doesn't do rendering, and may not want to include
Colorspace conversion machinery or HDR colorspace/conversion machinery).

> Nothing prevents adding more protocol to give apps more hints to behave
> more optimally wrt. to the compositor's internal pipeline.

There's a difference between hints that help speed and quality but
still burden the compositor with doing display dependent conversions,
and an approach that removes hat burden completely to the client when
it does the rendering.

> Correct, and Shashank is not proposing anything different. The
> per-channel lookup curves are a compositor internal detail, always
> programmed by the compositor correctly according to what it happens to
> be showing each monitor refresh on that specific monitor.

Since the compositor is not Color Management aware, then by definition
it can't set the CRTC to the correct values ("correct" in terms
of the color sensitive end users intentions for how they need their systems
and applications to work.) Now if the compositor was CM aware, it could
choose whether to implement CM display calibration curves by using
the hardware CRTC, or by implementing it in some other fashion such
as with a shader, or (ideally) by downloading the curve to the display
itself (many high end displays have this capability, and the
commercial CM tools do exactly that. How that could work with
Wayland is another problem to ponder.)

Cheers,
	Graeme Gill.