[RFC wayland-protocols] Color management protocol

Carsten Haitzler (The Rasterman) raster at rasterman.com
Sun Dec 11 02:13:17 UTC 2016 

On Sat, 10 Dec 2016 09:50:53 +0100 Niels Ole Salscheider
<niels_ole at salscheider-online.de> said:

> On Saturday, 10 December 2016, 11:48:32 CET, Carsten Haitzler wrote:
> > On Fri, 09 Dec 2016 14:38:37 +0100 Niels Ole Salscheider
> > 
> > <niels_ole at salscheider-online.de> said:
> > > Am Donnerstag, 8. Dezember 2016, 15:51:12 CET schrieb Graeme Gill:
> > > > Niels Ole Salscheider wrote:
> > > > > Therefore I think that the situation has changed and I'd like to
> > > > > propose
> > > > > this protocol for inclusion in wayland-protocols again.
> > > > > What do you think?
> > > > 
> > > > Hi,
> > > > 
> > > > 	I'm prompted to look into the current state of color management
> > > > 
> > > > in Wayland, by Richard Hughes comment on the ArgyllCMS mailing list
> > > > 
> > > > recently that:
> > > > > About this; in the near future systems will be migrating from X11 to
> > > > > Wayland (Fedora 25 already defaults to Wayland, other distros will
> > > > > follow) and so setting X atoms is no longer going to work. Even with
> > > > > XWayland (the compatibility "wrapper" that provides an isolated
> > > > > xserver for the app) you can't use the root window as it's isolated
> > > > > from the other windows. I think most applications that want to know
> > > > > what profile to use are now using either libcolord, or more commonly,
> > > > > the colord DBus API.
> > > > 
> > > > What I take from this is that XWayland is lacking in its emulation
> > > > of existing X11 color management protocols (primarily due to lack
> > > > of underlying support in Wayland), and that currently the only
> > > > option for pure Wayland applications is to depend on system
> > > > specific work-arounds such as using Weston with colord. I can
> > > > therefore see that users that depend on color managed X11 applications
> > > > (such as photographers, desktop publishers, video editors etc.)
> > > > aren't going to be switching to Wayland based systems any
> > > > time soon.
> > > > 
> > > > Looking through the current Wayland color-management protocol
> > > > proposal, I think it is missing a really fundamental thing -
> > > > managing the output device color calibration state and color
> > > > profile. I guess the assumption is that this is being done
> > > > by colord, but my understanding is that colord is specific
> > > > to Gnome based systems, and certainly depends on DBus, which
> > > > Wayland does not. [ Please correct me if I've got any of this wrong. ]
> > > 
> > > It is (currently) up to the compositor to decide how to implement this.
> > > The
> > > compositor could come with its own settings for the output color profiles
> > > or query some other program. This might be colord, but it could also be
> > > kolormanager, or something else.
> > > 
> > > > It also seems fundamentally poor design to be using a parallel
> > > > protocol to manage the color of the graphics system, rather
> > > > than it being kept in sync with the elements being managed,
> > > > such as outputs and pixel rasters, etc. Certainly in X11 it is
> > > > all kept within the X11 protocol or its extensions.
> > > > If Wayland gets extended to be a remote protocol, then
> > > > the existence of in band protocols for color management
> > > > become even more important.
> > > 
> > > Yes, with the protocol I proposed there is a (small) time window after
> > > changing the output color space where the application might still display
> > > surfaces with the old color space...
> > 
> > wouldn't it be best not to explicitly ask for an output colorspace and just
> > provide the colorspace of your buffer and let the compositor decide? e.g. if
> > your window is on top, or it's the largest one, or it's focused,then the
> > compositor MAY switch the colorspace of that monitor to match your
> > surface's buffer colorspace, and if it goes into the background or
> > whatever, switch back? it can (and likely should) emulato other colorspaced
> > then.
> 
> I think there is a misconception here. For normal applications it would work 
> like this:
> surface color space -> blending colorspace (where the compositor does 
> blending) -> output color space.
> If the application is fullscreen then the compositor does not need to do 
> blending and can just do the surface color space -> output color space 
> conversion.

actually... all the major toolkits and apps work in ARGB premul (which is SRGB
with gamma 2.2 or whatever - the non-linear one). they actually work natively
in that space so they are producing what i have, for short, called "sRGB
buffers" (sARGB premul?). so this is currently the native colorspace for all
wayland compositors, client buffers AND for actual monitors.

what we're talking about here is clients providing buffers in ANOTHER RGB
colorspace. this is exactly the same as YUV (actually there's BT601 and BT709
YUV/YCbCr colorspace depending on SD vs HD, and in fact there's also now a
BT2020 for UHD. in fact this is a problem in wayland right now as there way i
know of to distinguish the colorspace of a YUV buffer - is it 601, 709 .. or
2020?

this is the same issue with RGB colorspaces. totally the same. the difference
is that the various YUV colorspaces are far more widespread than specialized
RGB colorspaces. in the end generally with yuv it either is handedoff to yuv
plane hardware to deal with to convert to regular RGB output signal or it's
converted with software or fragment shaders. no matter what, unless you have
hardware plane support you need to do a conversion which can be "not that
cheap" if you have to do it on the cpu.

> But Graeme talked about professional applications that want to do color 
> management on their own because they have more complex needs.

why do they want to MANAGE color. all they need is to PRESENT their content in
their preferred colospace. let's think adobe RGB. if photoshop/gimp/whatever
has an image that is set to be in this colorspace then it can either switch its
entire window to adobe rgb (and appropriately alter controls/button/icons and
what not to look right) or use a subsurface just for the image in this
colorspace.

*IF* the monitor is adobe rgb capable, the compositor can pass through this
surface buffer without modification. 1:1 pixel values to the framebuffer (or
assign to a hw layer maybe). perhaps the screen has to be configured to be in a
different colorspace and the compositor can do this at this time. if this is
not possible then the compositor will have to downconvert the adobe rgb data to
sRGB for display. the client needs not CONTROL over the screen. the compositor
handles that.

this is also no different to a client providing 24/32bpp buffers but the screen
being 18bpp, 16bpp, 15bpp or 8bpp or pure 1bit monochrome.

the compositor will downconvert to these color formats and dither or whatever.

of course this relies on the compositor supporting that colorspace. so all that
is really needed is:

1. list of colorspaces supported (and which ones can be native to the screen)
2. the ability to tag a buffer with a colorspace (needed for yuv too as above).

there is no need for an application to know the current colorspace of the
screen - just what is possible (perhaps it can do 3 or 4 different ones? maybe
only 1? but no need to know the CURRENT one), and no need to directly CONTROL
he current colorspace. leave that to the compositor. just attach colorspace to
the buffer.

> These application would query the output color space of the screen that they 
> are currently on and do all conversions on their own.

just PROVIDE a buffer in that colorspace... what the app does to make this
happen is totally up tot he app itself... :)

> These applications would not want the compositor to do anything to the colors 
> and indicate this (implicitly) by setting the surface color space to the 
> output colorspace.

attach to the buffer not the surface. cleaner :) as the content of the buffer
has been rendered in that colorspace... right? :)

the compositor may still do things to the pixels in the buffer when drawing if
the colorspace of the screen at the time is not the same as the buffer
colorspace. :)

> > e.g. if buffer is adobe rgb, then switch display to work in adobe rgb but
> > re-render everything else that is sRGB into adobe argb space... there might
> > be a slight "flicker" so to speak as maybe some banding appears in some
> > gradients of SRGB windows or colors are ever so slightly off, but the
> > compositor is optimizing for the surface it thinks it most important. i
> > really don't like the idea of applications explicitly controlling screen
> > colorspace. simple being able to list colorspaces available, know which
> > might be native or emulated, and then say which colorspace their buffer
> > has. this way colorspace is tired directly to the buffer and the compositor
> > can avoid these glitches (like time difference between switching screen
> > colorspace and buffers actually being provided in that colorspace).
> > 
> > > > So as a broad outline, what I would regard as features of
> > > > a reasonable color management facility for a graphics
> > > > system such as Wayland are:
> > > > 
> > > > * That color management protocol's have two uses :- 1) configuring
> > > > 
> > > >   color management and 2) allowing applications to use color management.
> > > >   These two uses may need different security profiles.
> > > >   The assumption should be that color management applications
> > > >   used to create calibrations, profiles and configure the
> > > >   state of color management, are of equal importance to color
> > > >   managed applications that depend on a properly profiled and
> > > >   configured color management system, since you can't have latter
> > > >   with out the former.
> > > > 
> > > > * Color management of a graphics rendering system such as Wayland
> > > > 
> > > >   should be split into two levels (possibly two extensions) :-
> > > >   
> > > >   1) Core == output device management, which involves identifying output
> > > > 
> > > > devices, controlling their calibration state (Video LUT), installing a
> > > > color profile resources associated with that output device, and
> > > > identifying
> > > > which rendered pixels go to which output device(s). This is the most
> > > > basic
> > > > requirement, since this is the bare minimum for applications to be
> > > > properly
> > > > color managed. It is also a necessary resource for the second level to
> > > > be
> > > > able to operate.
> > > > 
> > > >   2) Enhanced == input space management, which assumes a graphics
> > > >   server rendering system capable of color management. This involves
> > > > 
> > > > labeling an input raster with a color profile, and controlling the
> > > > manner
> > > > of rendering (i.e. rendering intent, maybe blending space ? etc.) This
> > > > is
> > > > useful for implementing default color management (e.g. as a means of
> > > > coping
> > > > with wide gamut displays when many applications are not color aware), or
> > > > as
> > > > a low developer overhead means of implementing basic color management in
> > > > non color critical applications.
> > > > 
> > > > Translating these aims into a more concrete set of capabilities might
> > > > look like this:
> > > > 
> > > > Core:
> > > > 	Get corresponding CRTC regions for a given Surface.
> > > > 	Get corresponding Output(s) for a CRTC.
> > > > 	Get unique Monitor identifier for an Output (i.e. EDID)
> > > > 	Get/Set CRTC per channel lookup tables.
> > > > 	Clear/Set/Get ICC profile associated with an Output.
> > > > 	
> > > > 	Event notifications:
> > > > 	- CRTC to surface mapping change
> > > > 	- Output to CRTC mapping change
> > > > 	- Monitor to Output connection change
> > > > 	- ICC profile associated with Output change
> > > > 
> > > > Enhanced:
> > > > 	Clear/Set/Get default color management source RGB ICC profile
> > > > for a
> > > > 
> > > > display. Disable/Enable/UseDefault color management flag for a Surface.
> > > > 
> > > > 	Clear/Set/Get source RGB ICC profile for a Surface.
> > > > 	Set/Get source and destination rendering intents for a Surface.
> > > > 
> > > > Given that Wayland doesn't seem to have current support for configuring
> > > > CRTC's, Outputs etc., there still seem to be some big gaps to fill to
> > > > add
> > > > even core color management support as part of the Wayland protocol.
> > > > 
> > > > Graeme Gill.
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > _______________________________________________
> > > > wayland-devel mailing list
> > > > wayland-devel at lists.freedesktop.org
> > > > https://lists.freedesktop.org/mailman/listinfo/wayland-devel
> > > 
> > > _______________________________________________
> > > wayland-devel mailing list
> > > wayland-devel at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/wayland-devel
> > 
> > --
> > ------------- Codito, ergo sum - "I code, therefore I am" --------------
> > The Rasterman (Carsten Haitzler)    raster at rasterman.com
> > 
> > _______________________________________________
> > wayland-devel mailing list
> > wayland-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/wayland-devel
> 
> 
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/wayland-devel


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster at rasterman.com

More information about the wayland-devel mailing list