[RFC wayland-protocols] Color management protocol

[Kai-Uwe]

Am 20.12.2016 um 14:08 schrieb Pekka Paalanen:
> On Tue, 20 Dec 2016 13:05:10 +0100
> Kai-Uwe <ku.b-list at gmx.de> wrote:
> Oh nice. So indeed, CMS belongs in the compositor too (not only
> clients), because it is the window manager in the Wayland architecture.

The compositor has at least access to the gamma table API and output
dimension information, which matches Graemes base CM

> But yes, I did mean to include also political decisions on what belongs
> where.

To have the most common client surface/buffer ICC profile -> wayland
case covered is a good political signal.

> After thinking about my last reply to Graeme, I have become more
> convinced that compositors must be full-fledged CMS users, not only
> applications. Now the question becomes: what do you need from Wayland,
> so that the application side instance of CMS can relay the necessary
> information to the compositor-side CMS, so that the compositor can do
> the right thing? And vice versa.

base CM
* set/get graphics card gamma table curves
  * profiler and CMS
* set/get the compositor ICC device profile per output
  * profiler and CMS
* CM off-switch per client surface/buffer
  * useful for early color binding
  * profiling
* get output dimensions and notified about changes
  * help profilers with UI layout
  * early color binding
* get output EDID and notified about changes
  * identify monitor
  * preselect profiles in UIs for a monitor
  * configuration and online ICC Taxi DB
  * ICC profile generation from EDID

advanced CM:
* set document/surface/buffer source ICC profile
  * set and forget - great for most clients
* set document/surface/buffer <-> output ICC device link profile
  * very advanced
  * reduces even further the need for early color binding
  * movie tools like blender, video players etc.
  * use custom CMM with own gamut mapping algorithm
  * use effects
  * adapt to viewing environment ...
* set a CMS of choice inside the compositor to replace lcms if needed.

> When one integrates a CMS in a compositor, you no longer need to
> expose configuration (hardware configuration, like CLUT programming)
> via any protocol. The compositor talks directly with the CMS and if the
> compositor can set e.g. CLUTs, CMS can tell it what to set.

only for the content channels <= display channels and content precision
is <= surface precision

I doubt that we can always know, what is needed for a CMS hungry /
demanding client.

> I am assuming that the compositor can interface with a CMS by calling
> into a library offered by the CMS. If that interfacing was previously
> done over X11, then you have to write that library. It will be more
> efficient too, since you don't have to serialize and deserialize, and
> asynchronicity (problems) appear only when you want to.

Making this selectable similar to XDG desktop file selectors for DE's.
Is there a path to install such libraries into? What is the API to
interface with? I guess ArgyllCMS, Oyranos CMS and others might be
interested. DE integrators, administrators and users are switching those
on a regular basis, but need now to de-/install packages.

> I'm slowly starting to suspect that CMS designers need a slight paradigm
> shift: the compositor is meant to integrate with the CMS, instead of
> the CMS given low-level access to the hardware bypassing the window
> manager. CMS is no longer something that can be bolted on externally,
> like it is in X11. Embrace the idea of integration, and I belive and
> hope that it will pay off as a much more reliable architecture and
> polished user interfaces.

Agreed to the extent that clients can pass enough precision and channels
to the compositor CMS. (I was never able to handle e.g. CMYK, 6-channel
or HDR content inside the compositor.)

> Some of what used to go over X11 would
> probably be much better as a library ABI, but in X11 times it was not
> possible because X11 implied separate processes.
> 
> Btw. in X11, how do CMS integrate/interface with compositing managers?

KolorServer:
* kded daemon + D-Bus messages <-> KWin core
https://userbase.kde.org/Color_Management

CompICC:
* plug-in to Compiz, is part of the main process with access to
most(all?) internals
http://www.oyranos.org/FOSDEM2012/ColourManagementInCompositors2012.pdf

> Who does the colorspace etc. conversions?

ICC profiles -> CMM (lcms or others) -> 3D texture

> How do you control blending spaces?

ugly monitor color space blending, its a known limitation

> How have you implemented GPU-accelerated color mapping?

Yes, for instant and CPU relaxing color conversions, both of the above
implementations use shaders with OpenGL 3D textures.

For gamma 1.0 handling to work, the shaders need to accept in-and output
curves. Otherwise there is really ugly color banding as can be seen in
cameraRAW displaying through the above KDE/Compiz color servers.

> The compositor internal interfaces can and should be used for what
> Xinerama, RandR and whatever you have been using to configure an X
> server through X11. This time, the compositor needs to load and
> interface with the CMS.

I looked at weston code and found some glue. However the communication
from the compositor(and its CMS library) to the client is not clear to
me. People ask frequently about remote color management. So pretty soon
there is a demand for scattered CMS and clients for remote
configuration, conversions etc.

> As to what you really need from Wayland, there are two ways that are
> perhaps even complementary:
> 
> 1. The one Niels started with: a standard protocol extension that is
>    used manually.

I like approach for its shoot and forget as well as it integration feeling

> 2. The approach what e.g. libEGL uses: if you have a particular CMS
>    implementation, and the compositor initializes the CMS, the CMS can
>    hook up it's very own Wayland protocol extensions. When a client
>    initializes the same CMS, the CMS looks up the
>    CMS-implementation-specific extension from the Wayland display, and
>    uses it. This way everything about the Wayland protocol extension is
>    actually private to the CMS implementation. The cost is that you
>    need a library ABI in the CMS, one for compositors and one for
>    clients.

Interestingly you mention a way to extend Wayland privately. Is this a
good starter(?):
https://wayland.freedesktop.org/docs/html/ch04.html

That might be the path for many base CM features. The device link idea
could then be specified more independently from the wayland community as
it is _really_ advanced, I guess.

> A "benefit" of option 2 is that you don't have to go through the
> Wayland upstream review process but only your own.

Yes. That might accelerate things.

Still option 1. has the political benefit to make wayland embrace color
management by default. I doubt that any other group can have such a
impact. They can easily extent and complement.

Kai-Uwe