[RFC wayland-protocols v3 1/1] unstable: add color management protocol

Tue Apr 9 14:43:53 UTC 2019

On Fri, 5 Apr 2019 13:16:34 +1100
Graeme Gill <graeme2 at argyllcms.com> wrote:

> Pekka Paalanen wrote:
> 
> Hi,
> 
> > they are analogues. They are not transforms, instead they describe
> > properties of the image. Buffer scale is not a command for the
> > compositor scale the image, instead it describes the scale of the image
> > so that a compositor can present it in a suitable way anywhere. One
> > could say buffer scale is only half of a transform, just like a color
> > profile; source state.  
> 
> OK.
> 
> > The combining cannot be fixed at commit. It happens on composition.  
> 
> So I guess I'm wrong in understanding that commit triggers composition ?
> 
> <https://wayland.freedesktop.org/docs/html/ch04.html#sect-Protocol-Surface>:
> 
> "Once the client has finished writing pixels, it 'commits' the buffer; this permits the
> compositor to access the buffer and read the pixels. When the compositor is finished, it
> releases the buffer back to the client."
> 
> So how does this actually work ? (Is there documentation somewhere ?)

Hi Graeme,

triggers, yes, but a call to commit is not 1:1 equivalent to
composition. A good compositor composites at most once per monitor
refresh cycle. Doing it any more times would be a waste as the monitor
cannot show more frames in one cycle. A commit with damage schedules a
new composition (weston calls it repaint), but there may be many
commits and damages before composition actually happens.

Commits are not the only triggers for composition either, there are
compositor internal triggers too, like moving a window and animations.

Most often a composition is triggered by another window than the window
of "the client", when we think about protocol between the compositor and
the client.

It is also possible that a commit does not trigger a composition: if
the wl_surface does not contribute to the image on the monitor, there
is no need to composite.

The reason why the combining cannot be done exclusively on commit is
that the compositor may decide to present the window elsewhere without
waiting for the client to provide a new frame. It means the client
content stays the same, while the output it will be composited to
changes.

> > Let's consider a case where a client has committed an image (wl_buffer +
> > color properties etc.). That image is now in the wl_surface. The
> > wl_surface gets shown on output 1, for that the compositor creates the
> > necessary device link transformation and uses it. Then, the wl_surface
> > gets moved by the compositor and shown on output 2. The compositor
> > needs to create a new different device link transformation to do the
> > color mapping. Therefore the device link transformation cannot be fixed
> > at commit time - not even if you generated one for each output, because
> > a new output could be hotplugged later.  
> 
> And I wasn't assuming so, but I was assuming commit = compose, and that
> the color transform would be determined at composition time. What
> concerns me is that a surface doesn't represent the pixel value state
> without explicit coordination from the client. This seems like
> a poor model of how things actually are, and I'd really like
> to get this right.

Sorry, I don't think I understood your concern.

Color transform indeed shall be determined at composition time. It is
just that wl_surface.commit is something else, a way to latch in
multiple pieces of data atomically.

> > Right now there is no Wayland protocol object that would directly
> > represent "a frame of content" with all the properties it may have. It
> > could have been useful, but maybe not avoid the need for
> > wl_surface.commit because the set of needed state varies and new
> > extensions will need a point to hook their state updates to.  
> 
> Sure. But a color profile is not a property that is divorced
> from the pixel values - it is intimately tied to them in a similar
> way to pixel encoding, width & height or stride. Yet these are not
> sent to the surface as separate values, which would be the case if
> a wl_buffer was just a pointer to a chunk of shared memory.

You're right, a wl_buffer could be just a handle to memory and a size
and nothing else, and we could carry the other properties as wl_surface
requests.

All this is an artefact of how the protocol has evolved over time under
the stable ABI constraints.

> 
> [ If color profile is a property of the wl_surface, how does the
>   screencopy protocol communicate the colorspace of the wl_buffer
>   it returns ? It couldn't really return a wl_surface could it ? ]

No, it would offer a way to instantiate a color_space object related to
the captured image instance.

> > A
> > wl_buffer is only a part of content - more of an artifact of memory
> > management, and wl_surface is more than just the content... or is it?  
> 
> I think I understand, but it seems that a wl_buffer is not quite
> a chunk of memory, since it has graphic attributes as part of it,
> and not quite a complete description of what the pixels mean.

Indeed. At the time wl_buffer was invented, it was a complete
description - as far as people at the time were interested and aware of.

Then we found more properties we need, but also realized we cannot
extend wl_buffer anymore.

The wl_surface/wl_buffer split does match OpenGL and EGL APIs though.
OpenGL does not know or care what color space you are using, you just
tell it how to compute pixels. It does care about buffer pixel format
and width and height.

> > The EGL problem is in the client side: a client cannot attach anything
> > to a wl_buffer if it uses EGL Wayland platform, because it cannot get
> > to the wl_buffer at all.  
> 
> Right, but the server would attach the color profile to the wl_buffer
> on the clients behalf.

Yes and no. No, because the server cannot do that, it needs the client
to tell it what to do, but a client cannot tell it because it cannot
say directly which wl_buffer it applies to. Yes, because a client can
tell the server through a wl_surface which wl_buffer it refers to: the
one attached to the wl_surface.

> 
> So:
> 
> 1) Is it reasonable to tag a wl_buffer with the color profile to make it
>    a more complete description of the pixel values, and implement a more
>    natural model of how things are ?

No, if I understand what you mean: attaching a wl_buffer to a
wl_surface, then committing a color profile on that wl_surface would
actually tie the color profile to the wl_buffer.

That would become very confusing, because the assignment is indirect.
Clients that manage their own buffers could probably handle it, but
clients using EGL cannot avoid attaching a color profile every single
commit, because the EGL implementation may be re-using wl_buffer which
had a wrong color profile assigned before. EGL does not even have a
concept of color profiles, so transplanting that via Wayland would be
certainly novel in a bad way, in that simply using EGL API might change
the color profile.

Such indirect assignment is a pattern that has never been used in a
Wayland extension before as far as I know, so it would be quite
surprising, especially through wl_surface which has never done anything
like that before.

wl_surface is also enshrined in EGL Wayland platform API, so we cannot
just have EGL commit wl_buffers to something else without changing
everything.

> 2) How would this be implemented, since (AFAIK) the server wl_buffer structure
>    is assumed to be private to the back end implementation ?
>    (That's something I have a couple more more ideas about though.)

I think it would be possible, in the callback that handles
wl_surface.commit. It would look at the attached wl_buffer if any
(represented as wl_resource which is the real C data type), check if
the wl_resource destroy signal already contains a listener, and if not,
allocate a new wl_buffer associate object with the color profile and
register a wl_resource destroy listener. That way one could attach new
data into a wl_buffer object and look it up on demand, even without
access to the buffer factory.

Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20190409/fbd340da/attachment.sig>