DRM Format Modifiers in v4l2

[Rob Clark]

On Fri, Sep 1, 2017 at 3:13 AM, Laurent Pinchart
<laurent.pinchart at ideasonboard.com> wrote:
> Hi Nicolas,
>
> On Thursday, 31 August 2017 19:12:58 EEST Nicolas Dufresne wrote:
>> Le jeudi 31 août 2017 à 17:28 +0300, Laurent Pinchart a écrit :
>> >> e.g. if I have two devices which support MODIFIER_FOO, I could attempt
>> >> to share a buffer between them which uses MODIFIER_FOO without
>> >> necessarily knowing exactly what it is/does.
>> >
>> > Userspace could certainly set modifiers blindly, but the point of
>> > modifiers is to generate side effects benefitial to the use case at hand
>> > (for instance by optimizing the memory access pattern). To use them
>> > meaningfully userspace would need to have at least an idea of the side
>> > effects they generate.
>>
>> Generic userspace will basically pick some random combination.
>
> In that case userspace could set no modifier at all by default (except in the
> case where unmodified formats are not supported by the hardware, but I don't
> expect that to be the most common case).
>
>> To allow generically picking the optimal configuration we could indeed rely
>> on the application knowledge, but we could also enhance the spec so that
>> the order in the enumeration becomes meaningful.
>
> I'm not sure how far we should go. I could imagine a system where the API
> would report capabilities for modifiers (e.g. this modifier lowers the
> bandwidth, this one enhances the quality, ...), but going in that direction,
> where do we stop ? In practice I expect userspace to know some information
> about the hardware, so I'd rather avoid over-engineering the API.
>

I think in the (hopefully not too) long term, something like
https://github.com/cubanismo/allocator/ is the way forward.  That
doesn't quite solve how v4l2 kernel part sorts out w/ corresponding
userspace .so what is preferable, but at least that is
compartmentalized to v4l2.. on the gl/vk side of things there will ofc
be a hardware specific userspace part that knows what it prefers.  For
v4l2, it probably makes sense to sort out what the userspace level API
is and work backwards from there, rather than risk trying to design a
kernel uapi that might turn out to be the wrong thing.

BR,
-R