Any protocol documentation about premultiplied alpha?

Wed Sep 29 13:23:27 UTC 2021

Thanks, Pekka/Carsten/Simon.

I consider myself duly corrected ;)

Apparently my snapshot of Weston wasn't rebased after the master -> main transition, that shader fragment file didn't exist, and I didn't encounter the comment. I admit I didn't recognize the glBlendFunc() clues that already existed.

Would anybody be interested in some kind of comment inserted into the documentation for wl_buffer that expresses this implicit expectation?

-Matt

On 9/29/21, 4:38 AM, "Pekka Paalanen" <ppaalanen at gmail.com> wrote:

    On Tue, 28 Sep 2021 13:16:36 +0000
    "Hoosier, Matt" <Matt.Hoosier at garmin.com> wrote:

    > Hi there,
    >
    > Is there any definitive word about whether or not clients should be
    > expected or allowed to premultiply alpha the alpha channels in the
    > normal EGL and linux-dmabuf buffers they send to a compositor?

    Hi,

    yes, this is really badly undocumented. I think the convention may have
    been adopted from Cairo, as Cairo was probably the first drawing
    library used with Wayland.

    The Wayland rule is that everything is premultiplied if an alpha
    channel exists, and no protocol is saying otherwise. Currently,
    there is no protocol defined AFAIK that could say otherwise.

    https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/14
    has the intention to add a way for clients to say the pixels use
    straight alpha instead of premultiplied, because premultiplied pixels
    are harmful for color management. Color space transformations require
    straight alpha.

    > I have never seen this practice used with Wayland -- the shaders in
    > the gl-renderer certainly don't seem to expect clients to
    > premultiply.

    Weston's GL-shader does fully expect all client pixels to be
    premultiplied. Look for glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA); -
    this is what encodes it and the fragment shaders are mainly pass-through
    for client pixels.

    Also, if you look at fragment.glsl in Weston's main branch, you can see
    comments about premult having appeared in preparation for color
    management.

    > But I stumbled on a comment or two that maybe
    > gnome-calculator [1] does this and that Sway settled on that
    > convention [2].
    >
    > I see hardly any mention of premult in the Weston codebase. In fact,
    > the only real reference to it occurs at a site dealing with Pixman
    > color formats. In this location, it's a prominent warning to the
    > reader that Pixman does use premultiplication; this seems to carry
    > the meaning that Pixman's premult usage stands apart from the rest of
    > Weston.
    >
    > To me, it seems that Wayland's adoption of the DRM_FORMAT_xxx
    > identifiers in the wire protocols (e.g., dmabuf) combined with
    > drm_fourcc.h's complete stonewalled silence on the subject, seems to
    > be a fairly strong indicating that premultiplication should not be
    > used.

    KMS actually has a property to control the blending mode: "pixel blend
    mode", which when supported, changes the handling of pixel values.
    https://www.kernel.org/doc/html/latest/gpu/drm-kms.html#plane-composition-properties

    Unfortunately, I could not find a mention of which blend mode is
    assumed if the property is not exposed.

    Premult is not a property of the pixel format, hence drm_fourcc.h is
    correct to be silent about it. Different APIs can make their
    assumptions though.

    > [1] https://github.com/swaywm/wlroots/issues/984#issue-324334642
    > [2]
    > https://github.com/swaywm/wlroots/issues/984#issuecomment-390221778

    The rest below might be more than you asked for. ;-)

    On Tue, 28 Sep 2021 22:31:42 +0100
    Carsten Haitzler <raster at rasterman.com> wrote:

    > premultiplpied is the base assumption. you don't need shaders to deal with it -
    > the gl blend func does... the only time you want a shader to deal with
    > "premultiplied" is if the source buffer is NOT premultiplied - you now have to
    > do the extra multiply in-shader (or switch blend func). premultiplied is the
    > sane colorspace for compositing/rendering where things just work out nicely.

    Now that we are talking on the subject, there are actually *two*
    different ways to do premultiplied alpha:
    http://ssp.impulsetrain.com/gamma-premult.html

    - multiply alpha into non-linear (gamma encoded) color values
    - multiply alpha into light-linear color values and gamma-encode the
      result for storage/transmission

    One may not be aware that when alpha denotes coverage (not
    translucency, see http://ssp.impulsetrain.com/translucency.html ),
    blending must be done with light-linear values (gamma encoding removed)
    to match the physical model of how coverage works.

    If you do blending with non-linear values, not only does the result have
    different brightness compared to the physically more correct result
    (text rendering says hi!), but it can also be of unexpected hue! There
    will be more or less subtle changes in color you can't quite figure out
    where it comes from, and in most practical cases they are also hard to
    realise as you probably do not know what color the content designer was
    aiming for. Or if you're just writing hex color values in code, you
    probably have no idea. One may also wonder why simple gradients look
    not quite expected. Then people adjust the source colors until the
    result looks good enough.

    I would bet that all Wayland compositors (Weston included) and most
    other software not written by color enthusiasts will do blending with
    non-linear values. That is, simply with the 8-bit RGB values as-is that
    people are used to playing with. So quite likely almost everything does
    and expects alpha to be multiplied into non-linear pixel values. After
    all, computing the power function twice for each channel in each pixel
    is a lot of computing to do.

    When you pre-multiply into non-linear RGB values, and you blend in
    non-linear values, the amount of computations is kept to a minimum.
    If you want to do coverage blending right, you get to undo premult,
    undo gamma encoding, multiply with alpha, and then blend. And finally
    gamma encode the result again for display.

    Since Cairo has so much accidental influence on Wayland color handling,
    what does it do about premultiplication? I'm not sure, but judging from e.g.
    https://gitlab.freedesktop.org/cairo/cairo/-/issues/2
    I would assume Cairo does the naive simple thing which is useful only
    if one is doing blending "wrong".

    (Btw. these things apply also to linear texture filtering. For
    filtering to work correctly, it has to be done in light-linear and
    premultiplied values, I believe. Hello, sRGB textures, careful with the
    type of premultiplication there.)

    Also, I did not even get to color spaces yet. All the above is assuming
    your usual, undefined, mostly likely kinda close to sRGB, color space
    everywhere. The gamma function mentioned can be the real sRGB EOTF, or
    a power function approximating it like in the Pedersen article. The
    reason gamma encoding is used is to map the 8-bit value space a little
    better to the human visual system sensitivity than what using
    light-linear values as-is would. IOW, it's a compression function.

    Thanks,
    pq

________________________________

CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.