HDR support in Wayland/Weston

[Graeme Gill]

Pekka Paalanen wrote:



> yes, this is a good concern. I think we might be needing an extension
> to tell the client which output to prioritise for. We already have an
> analogue of that in the Presentation-time extension: a wl_surface's
> timings can only be synchronised to one output at a time, so the
> presentation feedback events tell the client which output it was
> synchronised to on each frame.

Hello Pekka,

yes, that sounds like a very similar concern, and one that perhaps
could be leveraged or imitated.

(I was being a overly pessimistic in expecting only one output
 to be accurately color rendered - in fact a color aware application
 should be able to do color accurate rendering to all surfaces
 that are mapped to a single output if that output has an associated
 color profile. It would just be surfaces mapped to more than
 one output that would be compromised on all but one output.)

> I cannot imagine a case where the output for timings and the output for
> color would be different, but I also probably would not re-use the
> Presentation-time extension here. Instead, we would probably need to
> handle it in the color related extensions explicitly.

OK.

> Your explanation further below is very enlightening on what "rendering"
> is.

Thanks.

> The explanation of "intents" is an eye-opener to me. I guess the only
> thing I had been imagining was the Absolute Colorimetric method - that
> colors could be absolute.

Absolute Colorimetric is less commonly used that Relative Colorimetric
style intents, since people normally expect the image white to map to
device white, no matter what that may be for a particular device.
Absolute colorimetric is typically has more specialized use,
such as side by side proofing, spot colors etc.

>>> Yes, a compositor must implement all that, but this is now slipping to
>>> the topic of calibration, which is very much off-scope for today. We
>>> only want to consider how applications produce and provide content with
>>> specific color properties for now.

>> It's a necessary part of the picture. There's not much point in
>> moving ahead with Color Management support if there is no
>> easy means of creating display profiles to populate it with. So in
>> terms of practical implementation I see them going hand in hand.
> 
> They may go hand-in-hand in practise, but protocol-wise I still see
> them as two completely orthogonal features, and we need to split the
> work into manageable chunks anyway.

I don't have any basis to voice an opinion as to which particular protocol
these different aspects would best fall into (wayland core or one of the
supplementary xdg protocols ? - I'm not clear on what the purpose of these
divisions is), I just know that they are intimately related, and development
of one without the other will risk mis-steps. I don't really understand
why you think they are orthogonal.

> Surely the color characterisation of a monitor is not specific to a
> window system?

Very much so, since in practice this process relies on running an application
that makes use of the window system for its operation.

> While we still don't have a good solution to
> measurements in Wayland, people can measure their monitors on Xorg, or
> even Windows, I hope.

It's unsafe to switch pixel pipelines in the process of characterization,
since it risks measuring transformations that happen/don't happen in
one context, and not in the other. It's also not something that
end users will put up with - if a system claims to support color management,
then it includes the color management tools necessary to setup,
maintain and verify its operation on a day to day basis.

>From a development point of view it's working in a an unnecessary
difficult manner, since there will be nothing to exercise the full
range of color management capabilities, or measurably verify
the implementation against.

> Or even better: using a measuring tool running
> directly on DRM KMS, which ensures the tool gets absolute control over
> the display hardware.

Definitely the last thing you want, since that leaves a gaping
hole for the hardware to be setup differently between profiling
and use of the profile, thereby making the profile invalid.
The safe approach is to maintain as much similarity as possible
between system setup while profiling and when the profile
is used. Ensuring that a profile remains valid from
profiling to use is a key consideration in color management,
and without this, it all falls to pieces.

Completely made up example: Say the display is actually
a TV running over HDMI, and (for whatever reason) the
compositor sets up the link in YCbCr mode rather than RGB,
and as luck would have it, the video card encodes the framebuffer
RGB using REC 709 1150/60/2:1 standard, while the TV manufacturer
didn't bother supporting the various YCbCr encodings and hard
coded it to decode Rec601 YCbCr. So there is a hidden color
change going on when Wayland is running as well as some
minor quantization errors being introduced. But if the color
profiling application was to configure the hardware, it
may instead setup the HDMI in RGB mode (being the one
with the highest fidelity), and the color response won't
be the same as Wayland, so the profile made this way will
be invalid. Ensuring the graphics card and every other
element in the rendering pipeline is setup the same way,
by the same software, this hard to diagnose type of error is avoided.

Another aspects is that this raises considerably higher
barriers for translating the the profiling application onto wayland,
since it then has to bring it's own GUI/application layer to be able
to driver the display and get user input.

[ And consider that a full on color management application does
  a whole lot more that calibrate & profile displays, such as
  profiling printers, scanners, cameras, examining and previewing
  profiles images etc. etc. ]

> DRM Leases could make such a tool usable under a
> Wayland compositor, too, so you can have a control UI window as a
> Wayland window on another output. 

Doesn't that assume there is more than one output ?

Note that profiling tools typically provide a lot of
on screen guidance when making measurements, including where
to position the instrument, progress of patches etc.

See <https://www.youtube.com/watch?v=LFp-jBwMFFc>,
<https://www.youtube.com/watch?v=lXtSaLi9ZZE> etc. for a typical examples.

> People can take advantage of color
> managed output already while the community figures out a good way to
> handle measuring.

I'm quite not sure what you mean by that, since the actual measuring (i.e.
obtaining instrument values) is a solved problem. What has to be done
for Wayland is installing calibration curves and profiles. As long as
the profiling application can do that, then it can proceed with
calibration, profiling, installing and verification.

It's unnecessarily complicated (and more fraught with mistakes, see
above) to have some sort of special calibration mode. To send device
RGB to the display, all the profiling application has to do
is mark the source colorspace as being the same profile as the
corresponding output profile. To turn calibration curves off,
all it has to do is temporarily install a profile with linear per
channel curves. At the end of the process all it has to do is install
the computed profile & calibration curves permanently.
To verify calibration it just has to send device RGB values
through the installed calibration curves. To verify profile
conversion it just has to label its test RGB values with a known
source space profile. More complex from a Wayland point of view
appears to be getting control of the test window output, location
& size (although I get the impression some of this is doable using
xdg protocols ?)

> As an example of orthogonality, I would like to point to the new
> touchscreen calibration tool for Weston. Commit adding the new tool
> lists the benefits it has over the old calibration tool that was using
> the usual input interfaces in Wayland:
> https://gitlab.freedesktop.org/wayland/weston/commit/b79dead1dded6744d251da5357e14c83b91cef32

Right, but that seems to be an application that works through the compositor,
not around it ? Having a special mode to turn off color calibration curves
isn't necessary - installing linear per channel curves is all that's needed
and avoids extra (confusing) API's.

> The reason the extension is specific to Weston is that there was no
> interest in making it a standard, no-one else saw the need to calibrate
> touchscreen input. It could still be standardised if the interest
> arises.

Sure, and having a defined protocol and example implementation in Weston
is about all one can expect in the case of color management. Having
a working example and tools and applications that have been proven to
work correctly with the implementation provides a path for wider adoption
when demand arises.

> Likewise, measuring the color characteristics of a monitor will likely
> require a notion of intent to a Wayland compositor. Not only one wants
> to be absolutely sure the compositor is not mangling the pixel data,

Quite the contrary - see above. If the compositor is mangling all the
pixel data, then it needs to mangled in exactly the same way while
profiling so that the response is the same and so that the profile
is valid for the compositor mangling pixel data that way. To have
it un-mangled while profiling, yet mangled for all the applications
means that the profile is invalid.

Cheers,
	Graeme Gill.