[RFC 0/1] Color manager calibration protocol v1

Wed Apr 17 11:36:56 UTC 2019

On April 17, 2019 12:38:15 PM GMT+02:00, Erwin Burema <e.burema at gmail.com> wrote:
>On Wed, 17 Apr 2019 at 11:32, Pekka Paalanen <ppaalanen at gmail.com>
>wrote:
>>
>> On Tue, 16 Apr 2019 23:42:30 +0200
>> Erwin Burema <e.burema at gmail.com> wrote:
>>
>> > On Tue, 16 Apr 2019 at 17:03, Pekka Paalanen <ppaalanen at gmail.com>
>wrote:
>> > >
>> > > On Tue, 16 Apr 2019 13:33:02 +0000
>> > > Erwin Burema <e.burema at gmail.com> wrote:
>> > >
>> > > > On Tuesday, 16 April 2019, Pekka Paalanen wrote:
>> > > > > On Tue, 16 Apr 2019 13:11:06 +0200
>> > > > > Erwin Burema <e.burema at gmail.com> wrote:
>> > > > >
>> > > > > > On Tue, 16 Apr 2019 at 12:45, Pekka Paalanen
><ppaalanen at gmail.com> wrote:
>> > > > > > >
>> > > > > > > On Sun, 14 Apr 2019 12:57:47 +0200
>> > > > > > > Erwin Burema <e.burema at gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Without a way to calibrate/profile screens an color
>management
>> > > > > > > > protocol looses a lot of its value. So to add this
>missing feature I
>> > > > > > > > wrote the following protocol.
>> > > > > > > >
>> > > > > > > > The idea is that the calibration/profiling SW only sets
>the RGB
>> > > > > > > > triplet and then the compositor is responsible to draw
>a rectanglular
>> > > > > > > > region on the selected output screen, since not all
>calibration tools
>> > > > > > > > will be at the center of the screen a user should be
>able to modify
>> > > > > > > > the placement of this rectanglular region. Unless
>specified the
>> > > > > > > > monitor profile (if any) should not be applied but the
>GPU curve
>> > > > > > > > should, currently to set a new curve the calibration
>tool should
>> > > > > > > > generate a new ICC profile with the wanted curve in the
>VCGT tag (I
>> > > > > > > > am not sure if this is the best option but would make
>the most
>> > > > > > > > universal one). In the end after profiling the last
>uploaded ICC
>> > > > > > > > could then be saved (although a compositor is not
>required to honor
>> > > > > > > > the request in that case it should send the not saved
>error). If the
>> > > > > > > > compositor doesn't save or the connection with this
>protocol is
>> > > > > > > > broken the compositor should restore previous settings.
>> > > > > > >
>> > > > > > > Hi,
>> > > > > > >
>> > > > > > > I only took a very quick glance, but I do like where this
>design is
>> > > > > > > going. I'll refrain from commenting on wl_surface vs. not
>for now
>> > > > > > > though.
>> > > > > > >
>> > > > > > > Forgive me my ignorance, but why is the "GPU curve"
>needed to be a
>> > > > > > > custom curve provided by the client?
>> > >
>> > > Hi,
>> > >
>> > > ok, below we are finally getting to the point.
>> > >
>> > > > > Ok, you start with an identity curve and iterate. Why only
>the "GPU
>> > > > > curve" instead of a "full" color correction transformation?
>> > > > >
>> > > >
>> > > > Since we are trying to setup the "GPU curve" in this case a
>full
>> > > > transform would only get in the way.
>> > >
>> > > Why?
>> > >
>> >
>> > Good question!
>>
>> Hi Erwin,
>>
>> there seems to be two very different cases, and below we discuss one
>or
>> the other paragraph by paragraph, which might be confusing.
>>
>>
>>         Case 1: Calibrating and/or profiling the monitor only
>>
>> This case is what I see as controlling the pixel values on the wire,
>> and the identity LUT instead of "GPU curve" access. I explain why no
>> "GPU curve" in my opinion further below.
>>
>>
>>         Case 2: Calibrating and/or profiling the full color pipeline
>>                 from application to a specific monitor
>>
>> This would be using the normal compositor color mapping pipeline and
>> adjusting the full output calibration and profile to achieve a
>desired
>> color reproduction.
>>
>
>Case 2 does not exists you do NOT calibrate a specific application to
>monitor. Once you have an output profile you do NOT need to calibrate
>every application individual you just use a color management system to
>map an input profile to an output profile. Case 2 won't work for the
>same reason you can't calibrate/profile HDR screens that don't support
>direct access (via something like a Freesync2_Gamma22 display mode),
>it is hard to predict what the whole colormanager is going to do so
>making profiles that way is a bad idea.

I think he means the way you do profiling on X. simply writing values to a surface and measure the output. It measures not the display but the whole pipeline from the surface to the output. This is what Graeme wants.

Its not a bad idea if you have a single static pipeline but we want to use all resource we have available and pipelines change depending on what is being shown. If there is an error in one pipeline and it wasn't used when profiling this has gained you nothing. I think the only  reasonable thin to do is profile the actual monitor. Applying a lut at the end or not is another topic.

>>
>> These two are mutually exclusive approaches, so they have independent
>> rationales and protocol requirements. I am still not sure which one
>you
>> want. In some old discussions with Graeme I think he advocated for
>Case
>> 2, so that a user will actually get what he wants even with a faulty
>> compositor.
>>
>> While Case 2 might be good for the end user, it also aims to hide any
>> compositor bugs in their color pipelines, and makes the resulting
>> profile potentially invalid for other compositors.
>>
>> I'm arguing a lot in the below. It's not so much that I think the
>> proposal is wrong, I'm not knowledgeable enough to fully judge that.
>My
>> aim is to avoid blindly repeating the design of X11-based software
>> stacks in case there might be a better design now that we have a
>> display server that is expected to participate in color management
>and
>> do color mapping on behalf of apps. With X11 that was not really
>> possible until the advent of X11 compositing managers. It is such a
>> fundamental design change, that I really want to prod people to try
>to
>> see the new possibilities. I just hope I don't become a "difficult
>> person" because of that. :-)
>>
>
>The thing is the introduction of compositors doesn't really change
>anything color wise, it makes things a bit more complicated and might
>give an opportunity to not load the calibration curve in the videocard
>LUT which will give some interesting options with regard to hardware
>planes but that is about it. From this side it is not really a game
>changer.
>
>> > > The goal is to create a full color transform from some client
>chosen
>> > > color space into the output color space, so that we can determine
>the
>> > > color profile of the output. You might iterate only on the curve
>at
>> > > first, but do you not then iterate on the full transform as well
>to
>> > > ensure you get the result you want, at least to check once? How
>will do
>> > > that with your extension?
>> > >
>> >
>> > No here we are not interested in the full transform that would be
>to
>> > limited we are only interested in the output part, especially we
>are
>> > interested in how a certain RGB triplet send to the display maps to
>> > X'Y'Z' (which is an absolute color space used as one of the two
>> > options for the color connection space in the icc profile) the
>X'Y'Z'
>> > triplet will be provided by the colorimeter/spectrometer which
>measure
>> > the patch. This of course means that before we have a profile we
>have
>> > to iterate over many patches. My extension is meant to be able for
>> > calibration software to set a certain RGB triplet for the patch
>after
>> > which the software will read out the X'Y'Z' value from the
>calibration
>> > tool (most likely via USB these days, although serial is also an
>> > option)
>>
>> This sounds like Case 1. In other words, you want to be able to set
>the
>> pixel values directly on the wire, so that only the monitor will
>affect
>> the measurement results.
>>
>
>Yes indeed
>
>> > The above is what is called characterization or profiling, another
>> > step that (optionally) happens before this is called calibration
>and
>> > is where we use the screen settings and the video card LUT to setup
>> > the display in a preferred state with a nice response if we have a
>> > really nice screen we might be able to get away with only
>calibrating
>> > and telling software it is sRGB/AdobeRGB in practice we need both
>> > (especially on cheaper screens).
>>
>> Screen hardware knobs sure, but the video card LUT I do not
>understand
>> why that (or the equivalent in case the compositor implements it with
>> code instead of hardware configuration) has to be controlled or
>> non-identity. See my further comments to the article by Elle Stone
>> explaining why I do not see the LUT as calibration.
>>
>> If you need to apply a curve for measurement purposes, why not do
>that
>> in the client before sending the pixel values to the compositor?
>>
>
>Traditionally it has always been seen as calibration for the exact
>reason you would probably need to ask Graeme since this is quite
>complex and I see that I haven't explained it well but don't really
>know how to explain it better.

Its obvious that it is calibration because it actually changes the color space. Its also obvious that its not calibrating the output.

IMO the vcgt tag solves an ux problem. You have to calibrate the output to get e.g. your desired white-point. Doing so by changing knobs on the output getting feedback, changing the knobs some more, etc, is not good ux. Ideally the computer would do that for us but it's not possible (apparently some monitors have some proprietary means to do so but realistically the computer can't) so as a workaround o improve the ux we just pretend that the vcgt is part of the output.

I don't think the vcgt tag is a problem. We create a pipeline with the two profiles and just add a lut to the end. Mapping he pipeline to actual hardware is still under compositor control.
>
>> > > Or asking in other words: why do you need the compositor to
>implement a
>> > > special-case path where it applies only a part of an ICC profile?
>> > >
>> >
>> > We are skipping the whole ICC profile, or at least the parts that
>will
>> > be actually used for color management, the VCGT tag is an optional
>> > non-standard extension useful to keep calibration and profiling
>> > information for a screen together. Color management will work fine
>> > without it (just need another way to store calibration information
>but
>> > that is a solvable problem)
>>
>> Yes. It is a special-case path. Usually the VCGT and the profile are
>> both applied by a compositor. This is very much the opposite of how
>X11
>> worked, and I believe the reasons why things on X11 worked like they
>do
>> is that there simply was no concept of the display server doing any
>> kind of color mapping. But on Wayland, we want the compositor to do
>the
>> color mapping and be good at it.
>>
>
>Except we are not working around anything at all and in fact this is
>the way MacOS X (which already has full screen color management build
>in) works! For wayland to be good at color management we need good
>output profiles for that we need this special-case path.
>
>> > > Why not let the compositor apply the output profile you submit in
>full,
>> > > and be smart about what you put in that profile?
>> > >
>> >
>> > Since we don't have an output profile yet? And this is all about
>> > creating one (+ calibration)
>>
>> Assuming you wanted to do full calibration and color profiling, I
>would
>> imagine:
>>
>> 1. Choose a base color profile in which you send you test patch color
>>    to the compositor.
>>
>> 2. Initialize the output color profile to the same profile as in step
>>    1. This will result in identity pass-through in the compositor.
>>
>> 3. Measure a set of colors.
>>
>> 4. Correct the output color profile according to measurement results.
>>
>> 5. If error between desired and actual measurement results is too
>>    large, go to step 3.
>>
>> What is wrong with this general idea?
>>
>
>If step 2 is a true identity transform (as it should be) it will work
>but adds a lot of extra complexity that we don't need for
>profiling/calibration. The iterative element would only be needed for
>calibration since nothing since characterization will be finished
>after measuring.
>
>> You can use it for all of calibrating, profiling, and verifying.
>>
>> Just that it is not characterising only the monitor, it will optimize
>> the output profile for a particular compositor in case the compositor
>> does something unexpected. It is Case 2.
>>
>
>Besides breaking the color management contract the whole resulting
>profile will be useless since the compositor is changing stuff under
>our feet it means we can't be sure if we profiled the whole display or
>not, or if things will be stable over time. It is the same reason why
>we can't profile/calibrate HDR screens without something like a direct
>access display mode.
>
>> However, if you leave out the iterative refinement, you'd get more or
>> less Case 1.
>>
>>
>> > > Are you trying to avoid having to tag the test color with a color
>> > > profile?
>> > >
>> >
>> > No, although making that optionally possible might be nice for
>verification.
>>
>> I think any verification should be done through the normal color
>> pipeline, as in, the public color management protocol interface used
>by
>> normal apps. But that requires using a wl_surface.
>>
>
>Probably true
>
>> > > To me it would make most sense if you picked one or the other:
>either
>> > > pass the test color values to the wire directly, which means you
>do not
>> > > play with the "GPU curve" as it will be identity; or use the full
>> > > existing color correction pipeline just like normal apps do, tag
>your
>> > > test color with a suitably manufactured color profile which
>results in
>> > > an identity transformation on the first iteration, and iterate on
>the
>> > > output profile you set.
>> > >
>> >
>> > But the calibration curve is strictly speaking not part of the
>profile
>> > and some screens really need that to create an at least somewhat
>> > usable profile, after setting the videocard LUT we of course need
>to
>> > characterize the output but as I said for that we don't need an
>input
>> > profile (just raw RGB triplets to the screen).
>>
>> Here is something very fundamental I just do not seem to understand.
>>
>
>And that is probably the reason why the rest is also quite hard to
>follow, since this is about the core idea. I just don't know how the
>explain it better then I already did :(
>
>> > > I think this is something Graeme tried to explain to me: using a
>> > > special path that does half of the operations is prone to bugs
>because
>> > > it does non-trivial things, affects the result of measurements,
>but is
>> > > normally not used. Although I think he would also object to the
>"pass
>> > > directly to the wire" mode as well, because that also is a
>special path
>> > > not used normally.
>> > >
>> >
>> > Graemme also said the following as author of Argyll:
>> > https://www.argyllcms.com/doc/calvschar.html
>>
>> Ok, I understand that difference between calibration and
>> characterization, and I also note that calibration is often
>> additionally stored in a profile file that characterizes a device.
>>
>> > Another good description comes from Elle Stone
>> >
>https://ninedegreesbelow.com/photography/monitor-profile-calibrate-confuse.html
>>
>> This one raises my point: "Altering the video card LUTs doesn't do
>> anything at all to the monitor itself."
>>
>> Therefore I do not consider the "GPU curve" as calibration. It is
>> something the computer software stack chooses, not something inherent
>> to or adjustable in the monitor hardware.
>>
>
>It is a bit of a workaround for "cheap" monitors that don't support
>internal LUTs, in an expensive (for example Eizo) monitor the LUT
>would be stored in the display.
>
>> In Wayland, the video card LUT is just a tool for the compositor to
>> change and use any way it wants at any time to achieve whatever the
>> compositor wants to do. It is not fixed by "calibration". Instead, it
>> will be used as part of the color transformation computed from input
>> and output color profiles. This idea is supported by video cards no
>> longer having a single LUT, but more complex color processing
>> pipelines, e.g. LUT-matrix-LUT. We want compositors to be able to use
>> that hardware instead of reserve it for "monitor calibration" or for
>> users to explicitly set.
>>
>> Elle's section A4 point 2 is relevant here.
>>
>> Is this idea of mine wrong?
>>
>
>Yes the more complex operations will be done in the actual GPU, the
>video card LUT is generally part of a CRTC and can't do anything more
>complex then apply 3 1D LUTS (one for each color channel) so all the
>complex stuff will happen before that. (In KMS terminology it is the
>GAMMA_LUT)
>
>> Is a monitor color profile, VCGT tag excluded, not capable of
>> describing a monitor in such way that one can compute the necessary
>> "GPU curve" just from the input and output color profiles and pixel
>> encodings?
>>
>
>Sort of yes, it is not really the right question tough, you could set
>the videocard LUT to identity and profile the screen that way, this
>will create a profile that effectively includes the "GPU Curve", and
>this can be used with any input profile without problems. Generally
>speaking it will be a lot less nice and would probably be a pretty
>large LUT based profile (since it isn't well behaved).
>
>> Why is the VCGT tag not simply a part of the color transformation
>> between PCS and the device color space? It seems like it is some
>> strange side-track that is arbitrarily used to mangle the color
>pipeline
>> output. A short-hand for something I'd expect would be computed from
>> the color profiles instead.
>>
>
>It was later unofficially added by I believe Apple who didn't want to
>deal with all kind of proprietary formats for so called "LUT loaders"
>(ArgyllCMS dispwin for example), theoretically you could make it part
>of the color transform (apply it after all the other color management
>stuff and then push the pixels to the screen) but even then according
>to the official terminology it would still be calibration (since it is
>still not part of the actual profile).
>
>> >
>> > So yes we do need a special direct pass and pass via video card LUT
>> > modes, and is another reason I prefer to not use a wl_surface for
>> > this, that would be to easily abused.
>>
>> I don't think it would not be any more easily "abused" that anything
>> else. The more complexity there is in a compositor, the more likely
>it
>> is to have bugs, if that is the "abuse". Adding a special pixel path
>in
>> addition to a normal pixel path is strictly increasing complexity.
>>
>
>Abuse in the sense of "placing a window in a weird location on tiling
>window managers", "bypassing color management", etc; so not so much
>about bugs but more that people can't use it for unintended use cases
>that might be detrimental to the experience.
>
>> > > > > > > Is the reason to use the "GPU curve" that you assume
>there is a
>> > > > > > > 8 bits per channel framebuffer and you need to use the
>hardware
>> > > > > > > LUT to choose which 8 bits wide range of the possibly 14
>bits
>> > > > > > > channel you want to address? (Currently a client cannot
>know if
>> > > > > > > the framebuffer is 8 bits or less or more.)
>> > > > > > >
>> > > > > >
>> > > > > > Currently not assuming anything about the curves bitnes, so
>not
>> > > > > > sure where you got that from? The curve should be set with
>the
>> > > > > > ICC vcgt tag I am pretty sure that supports higher bit
>depths
>> > > > > > then 8, if you have a better idea to be able to set the
>video
>> > > > > > card LUT I would like to hear that.
>> > > > >
>> > > > > If you do not assume anything about bitness, why does the
>protocol
>> > > > > spec have bitdepth? Is it the per-channel bitdepth of the
>> > > > > framebuffer, the wire, the monitor, the minimum for the whole
>pixel
>> > > > > pipeline, or?
>> > > >
>> > > > Maximum of whole pixel pipeline
>> > >
>> > > I don't see how that could be useful.
>> > >
>> >
>> > I think we have a bit of a misunderstanding going on here, and
>> > actually meant the same thing namely MIN( MAX(screen), MAX(GPU),
>> > MAX(communication) ), is that right?
>>
>> Right, so the minimum bit depth over the currently established pixel
>> path. Not sure taking the max of what each component might support is
>> more useful compared to what's actually used.
>>
>
>Ah yes of course.
>
>> > > > I tried to specified that the with a n-bit pipeline only the n
>least
>> > > > significant bits of the uint should be used. The intent is to
>only
>> > > > send 8 bit data over an 8 bit piline and the same for 10, 12 or
>14
>> > > > bits
>> > >
>> > > Oh, so the bitdepth value in the event is really just a scaling
>factor
>> > > for the test color values?
>> > >
>> >
>> > Sort of? 8bit would be sent as 0...0xxxxxxxx (so 24 zeroes followed
>by
>> > the actual 8 bit value), 10 bit would be 0...0xxxxxxxxxx (22 zeroes
>> > followed by 10 bit value), etc. I don't think that is scalling?
>>
>> Numerically it's just scaling, but let's call it encoding then.
>>
>> > > Ok, that explains why it is so vaguely defined and quite
>arbitrary.
>> > >
>> > > Why not use the full 32 bits mapping to [0.0, 1.0] always, let
>the
>> > > compositor convert it to whatever format the framebuffer uses,
>and
>> > > remove the event?
>> > >
>> >
>> > Will need to inspect the code of Argyll or get an answer from
>Graemme
>> > because this is a bit of an assumption of mine that we would need
>the
>> > actual display bitdepth, if we don't we can just as well do what
>you
>> > propose here.
>>
>> Right. If it's only a value encoding thing for the protocol, just use
>> 32 bits. If it is actually needed for the algorithms aside from
>simple
>> value encoding, then it will be needed for the algorithms, but the
>> protocol could still use just 32 bits. That would make the protocol
>> simpler to specify and understand.
>>
>
>Ah, I think I see what you mean now, yes that would work.
>
>>
>> Thanks,
>> pq
>_______________________________________________
>wayland-devel mailing list
>wayland-devel at lists.freedesktop.org
>https://lists.freedesktop.org/mailman/listinfo/wayland-devel

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20190417/f61e7035/attachment-0001.html>