DRM Format Modifiers in v4l2

[Brian Starkey]

On Mon, Aug 21, 2017 at 06:01:24PM +0200, Daniel Vetter wrote:
>On Mon, Aug 21, 2017 at 5:52 PM, Brian Starkey <brian.starkey at arm.com> wrote:
>> Hi all,
>>
>> I couldn't find this topic talked about elsewhere, but apologies if
>> it's a duplicate - I'll be glad to be steered in the direction of a
>> thread.
>>
>> We'd like to support DRM format modifiers in v4l2 in order to share
>> the description of different (mostly proprietary) buffer formats
>> between e.g. a v4l2 device and a DRM device.
>>
>> DRM format modifiers are defined in include/uapi/drm/drm_fourcc.h and
>> are a vendor-namespaced 64-bit value used to describe various
>> vendor-specific buffer layouts. They are combined with a (DRM) FourCC
>> code to give a complete description of the data contained in a buffer.
>>
>> The same modifier definition is used in the Khronos EGL extension
>> EGL_EXT_image_dma_buf_import_modifiers, and is supported in the
>> Wayland linux-dmabuf protocol.
>>
>>
>> This buffer information could of course be described in the
>> vendor-specific part of V4L2_PIX_FMT_*, but this would duplicate the
>> information already defined in drm_fourcc.h. Additionally, there
>> would be quite a format explosion where a device supports a dozen or
>> more formats, all of which can use one or more different
>> layouts/compression schemes.
>>
>> So, I'm wondering if anyone has views on how/whether this could be
>> incorporated?
>>
>> I spoke briefly about this to Laurent at LPC last year, and he
>> suggested v4l2_control as one approach.
>>
>> I also wondered if could be added in v4l2_pix_format_mplane - looks
>> like there's 8 bytes left before it exceeds the 200 bytes, or could go
>> in the reserved portion of v4l2_plane_pix_format.
>>
>> Thanks for any thoughts,
>
>One problem is that the modifers sometimes reference the DRM fourcc
>codes. v4l has a different (and incompatible set) of fourcc codes,
>whereas all the protocols and specs (you can add DRI3.1 for Xorg to
>that list btw) use both drm fourcc and drm modifiers.
>

This problem already exists (ignoring modifiers) in the case of any
v4l2 <-> DRM buffer sharing (direct video scanout, for instance).

I was hoping it would be possible to draw enough equivalency between
the different definitions to manage a useful subset through a 1:1
lookup table. If that's not possible then this gets a whole lot more
tricky. Are you already aware of incompatibilities which would prevent
it?

-Brian

>This might or might not make this proposal unworkable, but it's
>something I'd at least review carefully.
>
>Otherwise I think it'd be great if we could have one namespace for all
>modifiers, that's pretty much why we have them. Please also note that
>for drm_fourcc.h we don't require an in-kernel user for a new modifier
>since a bunch of them might need to be allocated just for
>userspace-to-userspace buffer sharing (e.g. in EGL/vk). One example
>for this would be compressed surfaces with fast-clearing, which is
>planned for i915 (but current hw can't scan it out). And we really
>want to have one namespace for everything.
>-Daniel
>-- 
>Daniel Vetter
>Software Engineer, Intel Corporation
>+41 (0) 79 365 57 48 - http://blog.ffwll.ch