[RFC wayland-protocols v2 0/1] Color Management Protocol

[Erwin Burema]

On Thu, 14 Mar 2019 at 14:04, Pekka Paalanen <ppaalanen at gmail.com> wrote:
>
> On Thu, 14 Mar 2019 13:27:45 +0100
> Erwin Burema <e.burema at gmail.com> wrote:
>
> > On Thu, 14 Mar 2019 at 12:29, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> > >
> > > On Wed, 13 Mar 2019 15:50:40 +0100
> > > Erwin Burema <e.burema at gmail.com> wrote:
> > >
> > > > Hi,
> > > > On Wed, 13 Mar 2019 at 10:06, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> > > > >
> > > > > On Tue, 12 Mar 2019 17:02:53 +0100
> > > > > Erwin Burema <e.burema at gmail.com> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Comments inline
> > > > > > On Tue, 12 Mar 2019 at 14:01, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> > > > > > >
> > > > > > > On Thu, 7 Mar 2019 12:37:52 +0100
> > > > > > > Erwin Burema <e.burema at gmail.com> wrote:
> > > > > > >
> > > > > > > > Wed Mar 6 17:09:27 UTC 2019 Sebastian Wick :
> > > > > > > > >...
> > > > > > > >
> > > > > > > > > 2. The whole pipeline should look something like
> > > > > > > > >
> > > > > > > > >   [surface cs] -cs conversion-> [output cs] -tone mapping-> [output cs]
> > > > > > > > >    -degamma-> [output linear cs] -blending-> [output linear cs] -gamma->
> > > > > > > > >    [output cs].
> > > > > > > > >
> > > > > > > > >    Where some parts can be skipped if e.g. surface cs == output cs or
> > > > > > > > >    surface and output are SDR.
> > > > > > > >
> > > > > > > > For expert/pro applications this is probably indeed the pipline
> > > > > > > > needed, the only thing that troubles me here is, is that there no
> > > > > > > > guarente that an output color space is well behaved so might still
> > > > > > > > contain some non-linearities or other issues after the degamma
> > > > > > > > process[1], so for non-expert/non-pro stuff that still cares to some
> > > > > > > > extent for color a better pipeline might be
> > > > > > > > [surface cs] -cs conversion-> [blending scene referred cs] -blending->
> > > > > > > > [blending scene referred cs] -tone mapping-> [blending output referred
> > > > > > > > cs] -cs conversion-> [output cs]
> > > > > > > > Of course since this requires two cs conversion steps this is not
> > > > > > > > ideal for anything pro but the only blending operation that is
> > > > > > > > guaranteed to be artifact free in output cs is bit blitting which is
> > > > > > > > not something we want to limit compositors to.
> > > > > > >
> > > > > > > Hi Erwin,
> > > > > > >
> > > > > > > did you consider that the blending we talk about here is only alpha
> > > > > > > blending with alpha in [0, 1] and 1-alpha coefficients and nothing
> > > > > > > else? I forget who recently noted that alpha blending shouldn't cause
> > > > > > > any problems since mathematically the result cannot exceed the source
> > > > > > > space.
> > > > > > >
> > > > > > > Could you elaborate on the potential problems with the proposed
> > > > > > > pipeline more?
> > > > > > >
> > > > > >
> > > > > > Yes alpha blending can't exceed color space but can lead to color
> > > > > > artifacts (if you ever edited/created a picture in sRGB and saw some
> > > > > > dark/black borders that is one possible effect)
> > > > > >
> > > > > > Some of the problems of blending colors in non-linear spaces are
> > > > > > described here:
> > > > > > https://ninedegreesbelow.com/photography/linear-gamma-blur-normal-blend.html
> > > > > > (not my site, has lots of pictures which hopefully makes the
> > > > > > explanation a bit better)
> > > > > >
> > > > > > Theoretically this should be solved by degamma but there is no
> > > > > > guarantee that after that operation a profile is well behaved (see
> > > > > > https://ninedegreesbelow.com/photography/well-behaved-profile.html),
> > > > > > it might be neither withe balanced nor normalized, in which case you
> > > > > > will get similar artifacts as if working in a non-linear space.
> > > > > >
> > > > > > > Do you mean that the degamma mapping computed from the output color
> > > > > > > profile might be so far off, that alpha blending would produce a
> > > > > > > visibly wrong color on the monitor?
> > > > > > >
> > > > > >
> > > > > > Yes, see above, also not sure if a degamma is always possible
> > > > > > (especially with LUT based profiles) Graeme Gill can probably give a
> > > > > > more complete answer here. Although this will be a bigger issue on
> > > > > > cheap consumer monitors then expensive pro-ones.
> > > > > >
> > > > > >
> > > > > > > I would lean on taking that risk though, to have support for "pro
> > > > > > > applications". People would definitely be upset if "pro apps" were not
> > > > > > > properly supported, while alpha blending is usually just for visual
> > > > > > > special effects like rounded corners or fade-in/out animations and
> > > > > > > other less important window system gimmicks. I also do not see the
> > > > > > > latter important enough to warrant implementing both ways in a
> > > > > > > compositor.
> > > > > > >
> > > > > > If you can be sure alpha blending is only used for effects and nothing
> > > > > > else this is an acceptable trade off but blending HDR and non-HDR
> > > > > > content should probably be done in a scene referred color space which
> > > > > > the output one isn't (even if it is degammaed) which means that in
> > > > > > some cases you might need/want the second way anyway, that said
> > > > > > pro-applications that deal with both HDR and non-HDR content should
> > > > > > probably take care of the critical parts of the blending itself and
> > > > > > output in HDR (so only the user interface would then be non-HDR)
>
> ...
>
> > > > Adding to this all I would like to note that although I think that for
> > > > the best possible output we need both a  critical and non-critical
> > > > color path I personally can live with some wonky alpha transparencies
> > > > considering that most of the time those should be limited to effects,
> > > > but I would hardly call that every frame perfect ;)
> > >
> > > Did the non-linearities from degamma not result purely because the
> > > output color profile was not good enough? That is, the ICC file is
> > > missing some optional parts or the data in them is not actually correct.
> > >
> >
> > No it comes from shitty cheap monitors, alas I am afraid that
> > explaining all this to the people who use those kind of monitors will
> > not be easy
>
> Hi Erwin,
>
> do you mean that the mapping from pixel values to light levels for a
> crappy monitor is not even expressable in the terms available in an ICC
> file?
>
> I mean, in terms that would allow one to go from the non-linear output
> space to a linear-light space?
>
> Or that the per-channel "gamma ramp" is not really invertible?
>
> Cross-talk between channels?
>
> Cross-talk between adjacent pixels?
>
> Something else too?
>

It will have mostly have non-proper white balance (so R=G=B=1 won't be
anything "white"), a curve per channel which might not be invertible
(so R=G=B=c won't have a constant hue/chroma for different c) and
cross talk between channels (in decreasing likelihood) all of which
can be described in an ICC file but it will most likely be a LUT
profile instead of a matrix one and a LUT profile can't be guaranteed
to be linear nor easily invertible.

> > > This may seem like finger-pointing, but if a compositor is given a less
> > > than perfect ICC profile for an output, there is no way it could ever
> > > make any frame perfect. Attempting to second-guess the profile will be
> > > futile in any case. I would not include the completely different
> > > blending space path, because it is trying to work around flawed data
> > > (the output color profile) in a way that will be wrong if the data was
> > > perfect.
> > >
> > > Weston has a policy to not work around driver bugs. I feel I can lump
> > > an inaccurate ICC file in the same heap.
> > >
> >
> > Not inaccurate just cheap monitors (it is accurate for the cheap
> > monitor it is just that cheap monitors have a tendency to be rather
> > wonky color wise), although you might consider that a HW bug
>
> If someone is using such a monitor, I wonder, do they actually benefit
> from color profiles at all? Isn't it just an arbitrary transform over
> an arbitrary presentation that might look nicer but have still a very
> poor correlation to colors on anything else?
>

They would benefit with a proper profile they would be able to get
much closer to proper colors then without in fact you can make a case
that these kind of people would benefit the most from a good profile
since a monitor that is already 99+% sRGB with the right kind of TRC
won't need as much correction, although making such a profile on such
a monitor is a bit harder (but this will be mostly hidden by good
profiling /calibration SW)

> The alternative would be to not use color management at all, which
> means that e.g. weston will blend in non-linear whatever space.
>
> If it seems like I am desperately trying to find an excuse to ignore
> this problem, that would be because I am. :-)
>
> But maybe I am mistaken and the problem is important. After all, I
> suppose crappy monitors are the huge majority.
>
Yes sadly they are and are also mostly used by people who don't fully
understand color management but still might want to benefit from it.
In the past most of these kind of issue where seen by professional
using monitors that could display in for example adobeRGB without full
desktop color management but at least to those you should be able to
explain why only their color managed apps looked OK while the rest
looked crappy, which is inverse of the case we have here with full
desktop color management and alpha blending. Of course we can just
ignore it and hope no one notices that the alpha blended colors look a
bit off

>
> Thanks,
> pq

Kind regards,

Erwin

PS. Very much enjoying this discussion hope I am not to much of a contrarian