[PATCH v4 01/10] drm/fourcc: Add AFBC yuv fourccs for Mali
Maarten Lankhorst
maarten.lankhorst at linux.intel.com
Mon Mar 18 10:17:55 UTC 2019
Op 12-03-2019 om 19:16 schreef Ayan Halder:
> From: Brian Starkey <brian.starkey at arm.com>
>
> As we look to enable AFBC using DRM format modifiers, we run into
> problems which we've historically handled via vendor-private details
> (i.e. gralloc, on Android).
>
> AFBC (as an encoding) is fully flexible, and for example YUV data can
> be encoded into 1, 2 or 3 encoded "planes", much like the linear
> equivalents. Component order is also meaningful, as AFBC doesn't
> necessarily care about what each "channel" of the data it encodes
> contains. Therefore ABGR8888 and RGBA8888 can be encoded in AFBC with
> different representations. Similarly, 'X' components may be encoded
> into AFBC streams in cases where a decoder expects to decode a 4th
> component.
>
> In addition, AFBC is a licensable IP, meaning that to support the
> ecosystem we need to ensure that _all_ AFBC users are able to describe
> the encodings that they need. This is much better achieved by
> preserving meaning in the fourcc codes when they are combined with an
> AFBC modifier.
>
> In essence, we want to use the modifier to describe the parameters of
> the AFBC encode/decode, and use the fourcc code to describe the data
> being encoded/decoded.
>
> To do anything different would be to introduce redundancy - we would
> need to duplicate in the modifier information which is _already_
> conveyed clearly and non-ambigiously by a fourcc code.
>
> I hope that for RGB this is non-controversial.
> (BGRA8888 + MODIFIER_AFBC) is a different format from
> (RGBA8888 + MODIFIER_AFBC).
>
> Possibly more controversial is that (XBGR8888 + MODIFIER_AFBC)
> is different from (BGR888 + MODIFIER_AFBC). I understand that in some
> schemes it is not the case - but in AFBC it is so.
>
> Where we run into problems is where there are not already fourcc codes
> which represent the data which the AFBC encoder/decoder is processing.
> To that end, we want to introduce new fourcc codes to describe the
> data being encoded/decoded, in the places where none of the existing
> fourcc codes are applicable.
>
> Where we don't support an equivalent non-compressed layout, or where
> no "obvious" linear layout exists, we are proposing adding fourcc
> codes which have no associated linear layout - because any layout we
> proposed would be completely arbitrary.
>
> Some formats are following the naming conventions from [2].
>
> The summary of the new formats is:
> DRM_FORMAT_VUY888 - Packed 8-bit YUV 444. Y followed by U then V.
> DRM_FORMAT_VUY101010 - Packed 10-bit YUV 444. Y followed by U then
> V. No defined linear encoding.
> DRM_FORMAT_Y210 - Packed 10-bit YUV 422. Y followed by U (then Y)
> then V. 10-bit samples in 16-bit words.
> DRM_FORMAT_Y410 - Packed 10-bit YUV 444, with 2-bit alpha.
> DRM_FORMAT_P210 - Semi-planar 10-bit YUV 422. Y plane, followed by
> interleaved U-then-V plane. 10-bit samples in
> 16-bit words.
> DRM_FORMAT_YUV420_8BIT - Packed 8-bit YUV 420. Y followed by U then
> V. No defined linear encoding
> DRM_FORMAT_YUV420_10BIT - Packed 10-bit YUV 420. Y followed by U
> then V. No defined linear encoding
>
> Please also note that in the absence of AFBC, we would still need to
> add Y410, Y210 and P210.
>
> Full rationale follows:
>
> YUV 444 8-bit, 1-plane
> ----------------------
> The currently defined AYUV format encodes a 4th alpha component,
> which makes it unsuitable for representing a 3-component YUV 444
> AFBC stream.
>
> The proposed[1] XYUV format which is supported by Mali-DP in linear
> layout is also unsuitable, because the component order is the
> opposite of the AFBC version, and it encodes a 4th 'X' component.
>
> DRM_FORMAT_VUY888 is the "obvious" format for a 3-component, packed,
> YUV 444 8-bit format, with the component order which our HW expects to
> encode/decode. It conforms to the same naming convention as the
> existing packed YUV 444 format.
> The naming here is meant to be consistent with DRM_FORMAT_AYUV and
> DRM_FORMAT_XYUV[1]
>
> YUV 444 10-bit, 1-plane
> -----------------------
> There is no currently-defined YUV 444 10-bit format in
> drm_fourcc.h, irrespective of number of planes.
>
> The proposed[1] XVYU2101010 format which is supported by Mali-DP in
> linear layout uses the wrong component order, and also encodes a 4th
> 'X' component, which doesn't match the AFBC version of YUV 444
> 10-bit which we support.
>
> DRM_FORMAT_Y410 is the same layout as XVYU2101010, but with 2 bits of
> alpha. This format is supported with linear layout by Mali GPUs. The
> naming follows[2].
>
> There is no "obvious" linear encoding for a 3-component 10:10:10
> packed format, and so DRM_FORMAT_VUY101010 defines a component
> order, but not a bit encoding. Again, the naming is meant to be
> consistent with DRM_FORMAT_AYUV.
>
> YUV 422 8-bit, 1-plane
> ----------------------
> The existing DRM_FORMAT_YUYV (and the other component orders) are
> single-planar YUV 422 8-bit formats. Following the convention of
> the component orders of the RGB formats, YUYV has the correct
> component order for our AFBC encoding (Y followed by U followed by
> V). We can use YUYV for AFBC YUV 422 8-bit.
>
> YUV 422 10-bit, 1-plane
> -----------------------
> There is no currently-defined YUV 422 10-bit format in drm_fourcc.h
>
> DRM_FORMAT_Y210 is analogous to YUYV, but with 10-bits per sample
> packed into the upper 10-bits of 16-bit samples. This format is
> supported in both linear and AFBC by Mali GPUs.
>
> YUV 422 10-bit, 2-plane
> -----------------------
> The recently defined DRM_FORMAT_P010 format is a 10-bit semi-planar
> YUV 420 format, which has the correct component ordering for an AFBC
> 2-plane YUV 420 buffer. The linear layout contains meaningless padding
> bits, which will not be encoded in an AFBC stream.
>
> YUV 420 8-bit, 1-plane
> ----------------------
> There is no currently defined single-planar YUV 420, 8-bit format
> in drm_fourcc.h. There's differing opinions on whether using the
> existing fourcc-implied n_planes where possible is a good idea or
> not when using modifiers.
>
> For me, it's much more "obvious" to use NV12 for 2-plane AFBC and
> YUV420 for 3-plane AFBC. This keeps the aforementioned separation
> between the AFBC codec settings (in the modifier) and the pixel data
> format (in the fourcc). With different vendors using AFBC, this helps
> to ensure that there is no confusion in interoperation. It also
> ensures that the AFBC modifiers describe AFBC itself (which is a
> licensable component), and not implementation details which are not
> defined by AFBC.
>
> The proposed[1] X0L0 format which Mali-DP supports with Linear layout
> is unsuitable, as it contains a 4th 'X' component, and our AFBC
> decoder expects only 3 components.
>
> To that end, we propose a new YUV 420 8-bit format. There is no
> "obvious" linear encoding for a 3-component 8:8:8, 420, packed format,
> and so DRM_FORMAT_YUV420_8BIT defines a component order, but not a
> bit encoding. I'm happy to hear different naming suggestions.
>
> YUV 420 8-bit, 2-, 3-plane
> --------------------------
> These already exist, we can use NV12 and YUV420.
>
> YUV 420 10-bit, 1-plane
> -----------------------
> As above, no current definition exists, and X0L2 encodes a 4th 'X'
> channel.
>
> Analogous to DRM_FORMAT_YUV420_8BIT, we define DRM_FORMAT_YUV420_10BIT.
>
> [1] https://lists.freedesktop.org/archives/dri-devel/2018-July/184598.html
> [2] https://docs.microsoft.com/en-us/windows/desktop/medfound/10-bit-and-16-bit-yuv-video-formats
>
> Changes since RFC v1:
> - Fix confusing subsampling vs bit-depth X:X:X notation in
> descriptions (danvet)
> - Rename DRM_FORMAT_AVYU1101010 to DRM_FORMAT_Y410 (Lisa Wu)
> - Add drm_format_info structures for the new formats, using the
> new 'bpp' field for those with non-integer bytes-per-pixel
> - Rebase, including Juha-Pekka Heikkila's format definitions
>
> Changes since RFC v2:
> - Rebase on top of latest changes in drm-misc-next
> - Change the description of DRM_FORMAT_P210 in __drm_format_info and
> drm_fourcc.h so as to make it consistent with other DRM_FORMAT_PXXX
> formats.
>
> Changes since v3:
> - Added the ack
> - Rebased on the latest drm-misc-next
>
> Signed-off-by: Brian Starkey <brian.starkey at arm.com>
> Signed-off-by: Ayan Kumar Halder <ayan.halder at arm.com>
> Reviewed-by: Liviu Dudau <liviu.dudau at arm.com>
> Acked-by: Alyssa Rosenzweig <alyssa at rosenzweig.io>
> ---
> drivers/gpu/drm/drm_fourcc.c | 16 ++++++++++++++++
> include/uapi/drm/drm_fourcc.h | 20 ++++++++++++++++++++
> 2 files changed, 36 insertions(+)
>
> diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c
> index 45c9882..a9df743 100644
> --- a/drivers/gpu/drm/drm_fourcc.c
> +++ b/drivers/gpu/drm/drm_fourcc.c
> @@ -225,6 +225,9 @@ const struct drm_format_info *__drm_format_info(u32 format)
> { .format = DRM_FORMAT_UYVY, .depth = 0, .num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1, .is_yuv = true },
> { .format = DRM_FORMAT_VYUY, .depth = 0, .num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1, .is_yuv = true },
> { .format = DRM_FORMAT_XYUV8888, .depth = 0, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1, .is_yuv = true },
> + { .format = DRM_FORMAT_Y210, .depth = 0, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 2, .vsub = 1, .is_yuv = true },
> + { .format = DRM_FORMAT_VUY888, .depth = 0, .num_planes = 1, .cpp = { 3, 0, 0 }, .hsub = 1, .vsub = 1, .is_yuv = true },
> + { .format = DRM_FORMAT_Y410, .depth = 0, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1, .has_alpha = true, .is_yuv = true },
> { .format = DRM_FORMAT_AYUV, .depth = 0, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1, .has_alpha = true, .is_yuv = true },
> { .format = DRM_FORMAT_Y210, .depth = 0, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 2, .vsub = 1, .is_yuv = true },
> { .format = DRM_FORMAT_Y212, .depth = 0, .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 2, .vsub = 1, .is_yuv = true },
> @@ -253,6 +256,19 @@ const struct drm_format_info *__drm_format_info(u32 format)
> { .format = DRM_FORMAT_P016, .depth = 0, .num_planes = 2,
> .char_per_block = { 2, 4, 0 }, .block_w = { 1, 0, 0 }, .block_h = { 1, 0, 0 },
> .hsub = 2, .vsub = 2, .is_yuv = true},
> + { .format = DRM_FORMAT_P210, .depth = 0,
> + .num_planes = 2, .char_per_block = { 2, 4, 0 },
> + .block_w = { 1, 0, 0 }, .block_h = { 1, 0, 0 }, .hsub = 2,
> + .vsub = 1, .is_yuv = true },
> + { .format = DRM_FORMAT_VUY101010, .depth = 0,
> + .num_planes = 1, .cpp = { 0, 0, 0 }, .hsub = 1, .vsub = 1,
> + .is_yuv = true },
> + { .format = DRM_FORMAT_YUV420_8BIT, .depth = 0,
> + .num_planes = 1, .cpp = { 0, 0, 0 }, .hsub = 2, .vsub = 2,
> + .is_yuv = true },
> + { .format = DRM_FORMAT_YUV420_10BIT, .depth = 0,
> + .num_planes = 1, .cpp = { 0, 0, 0 }, .hsub = 2, .vsub = 2,
> + .is_yuv = true },
> };
>
> unsigned int i;
> diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
> index 9fa7cf7..b992a38 100644
> --- a/include/uapi/drm/drm_fourcc.h
> +++ b/include/uapi/drm/drm_fourcc.h
> @@ -149,9 +149,13 @@ extern "C" {
> #define DRM_FORMAT_YVYU fourcc_code('Y', 'V', 'Y', 'U') /* [31:0] Cb0:Y1:Cr0:Y0 8:8:8:8 little endian */
> #define DRM_FORMAT_UYVY fourcc_code('U', 'Y', 'V', 'Y') /* [31:0] Y1:Cr0:Y0:Cb0 8:8:8:8 little endian */
> #define DRM_FORMAT_VYUY fourcc_code('V', 'Y', 'U', 'Y') /* [31:0] Y1:Cb0:Y0:Cr0 8:8:8:8 little endian */
> +#define DRM_FORMAT_Y210 fourcc_code('Y', '2', '1', '0') /* [63:0] Cr0:0:Y1:0:Cb0:0:Y0:0 10:6:10:6:10:6:10:6 little endian */
>
> #define DRM_FORMAT_AYUV fourcc_code('A', 'Y', 'U', 'V') /* [31:0] A:Y:Cb:Cr 8:8:8:8 little endian */
> #define DRM_FORMAT_XYUV8888 fourcc_code('X', 'Y', 'U', 'V') /* [31:0] X:Y:Cb:Cr 8:8:8:8 little endian */
> +#define DRM_FORMAT_VUY888 fourcc_code('V', 'U', '2', '4') /* [23:0] Cr:Cb:Y 8:8:8 little endian */
> +#define DRM_FORMAT_Y410 fourcc_code('Y', '4', '1', '0') /* [31:0] A:Cr:Y:Cb 2:10:10:10 little endian */
> +#define DRM_FORMAT_VUY101010 fourcc_code('V', 'U', '3', '0') /* Y followed by U then V, 10:10:10. Non-linear modifier only */
>
> /*
> * packed Y2xx indicate for each component, xx valid data occupy msb
> @@ -184,6 +188,15 @@ extern "C" {
> #define DRM_FORMAT_X0L2 fourcc_code('X', '0', 'L', '2')
>
> /*
> + * 1-plane YUV 4:2:0
> + * In these formats, the component ordering is specified (Y, followed by U
> + * then V), but the exact Linear layout is undefined.
> + * These formats can only be used with a non-Linear modifier.
> + */
> +#define DRM_FORMAT_YUV420_8BIT fourcc_code('Y', 'U', '0', '8')
> +#define DRM_FORMAT_YUV420_10BIT fourcc_code('Y', 'U', '1', '0')
> +
> +/*
> * 2 plane RGB + A
> * index 0 = RGB plane, same format as the corresponding non _A8 format has
> * index 1 = A plane, [7:0] A
> @@ -216,6 +229,13 @@ extern "C" {
> * index 0 = Y plane, [15:0] Y:x [10:6] little endian
> * index 1 = Cr:Cb plane, [31:0] Cr:x:Cb:x [10:6:10:6] little endian
> */
> +#define DRM_FORMAT_P210 fourcc_code('P', '2', '1', '0') /* 2x1 subsampled Cr:Cb plane, 10 bit per channel */
> +
> +/*
> + * 2 plane YCbCr MSB aligned
> + * index 0 = Y plane, [15:0] Y:x [10:6] little endian
> + * index 1 = Cr:Cb plane, [31:0] Cr:x:Cb:x [10:6:10:6] little endian
> + */
> #define DRM_FORMAT_P010 fourcc_code('P', '0', '1', '0') /* 2x2 subsampled Cr:Cb plane 10 bits per channel */
>
> /*
Hey..
There's a conflict with this patch and the merge of topic/hdr-formats, resulting in double definitions for Y210, Y410 and P010.
Worse still is that one has set has_alpha to true for Y41x and other to false.
~Maarten
More information about the dri-devel
mailing list