Any protocol documentation about premultiplied alpha?

Wed Sep 29 09:38:16 UTC 2021

On Tue, 28 Sep 2021 13:16:36 +0000
"Hoosier, Matt" <Matt.Hoosier at garmin.com> wrote:

> Hi there,
> 
> Is there any definitive word about whether or not clients should be
> expected or allowed to premultiply alpha the alpha channels in the
> normal EGL and linux-dmabuf buffers they send to a compositor?

Hi,

yes, this is really badly undocumented. I think the convention may have
been adopted from Cairo, as Cairo was probably the first drawing
library used with Wayland.

The Wayland rule is that everything is premultiplied if an alpha
channel exists, and no protocol is saying otherwise. Currently,
there is no protocol defined AFAIK that could say otherwise.

https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/14
has the intention to add a way for clients to say the pixels use
straight alpha instead of premultiplied, because premultiplied pixels
are harmful for color management. Color space transformations require
straight alpha.

> I have never seen this practice used with Wayland -- the shaders in
> the gl-renderer certainly don't seem to expect clients to
> premultiply.

Weston's GL-shader does fully expect all client pixels to be
premultiplied. Look for glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA); -
this is what encodes it and the fragment shaders are mainly pass-through
for client pixels.

Also, if you look at fragment.glsl in Weston's main branch, you can see
comments about premult having appeared in preparation for color
management.

> But I stumbled on a comment or two that maybe
> gnome-calculator [1] does this and that Sway settled on that
> convention [2].
> 
> I see hardly any mention of premult in the Weston codebase. In fact,
> the only real reference to it occurs at a site dealing with Pixman
> color formats. In this location, it's a prominent warning to the
> reader that Pixman does use premultiplication; this seems to carry
> the meaning that Pixman's premult usage stands apart from the rest of
> Weston.
> 
> To me, it seems that Wayland's adoption of the DRM_FORMAT_xxx
> identifiers in the wire protocols (e.g., dmabuf) combined with
> drm_fourcc.h's complete stonewalled silence on the subject, seems to
> be a fairly strong indicating that premultiplication should not be
> used.

KMS actually has a property to control the blending mode: "pixel blend
mode", which when supported, changes the handling of pixel values.
https://www.kernel.org/doc/html/latest/gpu/drm-kms.html#plane-composition-properties

Unfortunately, I could not find a mention of which blend mode is
assumed if the property is not exposed.

Premult is not a property of the pixel format, hence drm_fourcc.h is
correct to be silent about it. Different APIs can make their
assumptions though.

> [1] https://github.com/swaywm/wlroots/issues/984#issue-324334642
> [2]
> https://github.com/swaywm/wlroots/issues/984#issuecomment-390221778

The rest below might be more than you asked for. ;-)

On Tue, 28 Sep 2021 22:31:42 +0100
Carsten Haitzler <raster at rasterman.com> wrote:

> premultiplpied is the base assumption. you don't need shaders to deal with it -
> the gl blend func does... the only time you want a shader to deal with
> "premultiplied" is if the source buffer is NOT premultiplied - you now have to
> do the extra multiply in-shader (or switch blend func). premultiplied is the
> sane colorspace for compositing/rendering where things just work out nicely.

Now that we are talking on the subject, there are actually *two*
different ways to do premultiplied alpha:
http://ssp.impulsetrain.com/gamma-premult.html

- multiply alpha into non-linear (gamma encoded) color values
- multiply alpha into light-linear color values and gamma-encode the
  result for storage/transmission

One may not be aware that when alpha denotes coverage (not
translucency, see http://ssp.impulsetrain.com/translucency.html ),
blending must be done with light-linear values (gamma encoding removed)
to match the physical model of how coverage works.

If you do blending with non-linear values, not only does the result have
different brightness compared to the physically more correct result
(text rendering says hi!), but it can also be of unexpected hue! There
will be more or less subtle changes in color you can't quite figure out
where it comes from, and in most practical cases they are also hard to
realise as you probably do not know what color the content designer was
aiming for. Or if you're just writing hex color values in code, you
probably have no idea. One may also wonder why simple gradients look
not quite expected. Then people adjust the source colors until the
result looks good enough.

I would bet that all Wayland compositors (Weston included) and most
other software not written by color enthusiasts will do blending with
non-linear values. That is, simply with the 8-bit RGB values as-is that
people are used to playing with. So quite likely almost everything does
and expects alpha to be multiplied into non-linear pixel values. After
all, computing the power function twice for each channel in each pixel
is a lot of computing to do.

When you pre-multiply into non-linear RGB values, and you blend in
non-linear values, the amount of computations is kept to a minimum.
If you want to do coverage blending right, you get to undo premult,
undo gamma encoding, multiply with alpha, and then blend. And finally
gamma encode the result again for display.

Since Cairo has so much accidental influence on Wayland color handling,
what does it do about premultiplication? I'm not sure, but judging from e.g.
https://gitlab.freedesktop.org/cairo/cairo/-/issues/2
I would assume Cairo does the naive simple thing which is useful only
if one is doing blending "wrong".

(Btw. these things apply also to linear texture filtering. For
filtering to work correctly, it has to be done in light-linear and
premultiplied values, I believe. Hello, sRGB textures, careful with the
type of premultiplication there.)

Also, I did not even get to color spaces yet. All the above is assuming
your usual, undefined, mostly likely kinda close to sRGB, color space
everywhere. The gamma function mentioned can be the real sRGB EOTF, or
a power function approximating it like in the Pedersen article. The
reason gamma encoding is used is to map the 8-bit value space a little
better to the human visual system sensitivity than what using
light-linear values as-is would. IOW, it's a compression function.

Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20210929/b5932736/attachment-0001.sig>