[RFC] handling of alpha mode (pre-multiplied/straight) in ARGB modes

[Marek Szyprowski]

Dear All,

I would like to add support for configuring alpha modes: straight and 
pre-multiplied for blending to Exynos DRM driver. For those who see 
those terms for the first time:
1. straight alpha mode means means that all A and RGB values are from 
full 0.255 range,
2. pre-multiplied alpha mode means that A is from 0..255 range and RGB 
values are scaled to 0..ALPHA range (where ALPHA is the A value for the 
given pixel).
The pre-multiplied mode is quite common in texture processing.

I did a little research and found that there is no common approach for 
defining straight or pre-multiplied alpha modes for ARGB framebuffers.

 From reading the sources and the register names I found that following 
drivers use pre-multiplied modes for ARGB framebuffers:
radeon (at least for ARGB cursors),
amdgpu (cursor),
intel (all ARGB planes),
rock chip.

On the other hand, there are drm drivers which support ARGB modes and 
use straight alpha modes:
omap,
msm.

For the rest of drivers, which might deal with per-pixel alpha, I was 
not able to determine from the code if the pixel RGB values are 
interpreted as per-multiplied or straight:
atmel_hlcdc,
sti,
mdp5,
shmobile,
rcar,
vc4,
imx.

Exynos DRM driver now mixes pre-multiplied and straight modes, depending 
on the CRTC sub-driver.

While checking the code I found a following comment in 
drm/i915/intel_display.c in skl_plane_ctl_format() function:
/*
  * XXX: For ARBG/ABGR formats we default to expecting scanout buffers
  * to be already pre-multiplied. We need to add a knob (or a different
  * DRM_FORMAT) for user-space to configure that.
  */

The question is how to cleanup this ambiguities and let generic 
userspace to properly use ARGB/ARGB modes with proper blending mode. I 
came up with the following 3 solutions:

1. Define new fourcc values for pre-multiplied (or/and straight) alpha 
modes,
2. Introduce a new framebuffer flag for pre-multiplied or straight alpha 
(or both),
3. Introduce a new plane property for controlling alpha blending mode.

The first solution has serious compatibility problem, because we can 
either map existing fourcc to pre-multiplied (to follow radeon/amdgpu, 
intel and rockchip behavior) or straight mode. Both ways it will break 
some existing applications. To avoid compatibility problem we would need 
to add 2 more sets of fourcc and leave existing ARGB modes as 'driver 
dependent'.

The second solution is probably a bit less intrusive, but it suffers for 
the similar compatibility problem. To make this feature compatible with 
existing code, probably 2 flags will be needed: 
DRM_MODE_FB_ALPHA_FORCE_PREMULTIPLIED and 
DRM_MODE_FB_ALPHA_FORCE_STRAIGHT. This way when userspace provides no 
flag, the driver can use its default mode.

Third solution (additional plane property) has been already proposed: 
https://lkml.org/lkml/2015/6/19/330, however I found no follow-up nor 
comments. Separate property lets at least drivers to notify userspace 
about driver's default alpha mode and lets generic application to 
properly set the requested mode. The only problem is that the alpha mode 
is not directly configured on the framebuffer object, where in my 
opinion it should be stored.

Please let me know which approach You like best and which should I take 
for introducing generic way of configuring alpha mode.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland