DRM Format Modifiers in v4l2

[Daniel Vetter]

On Fri, Sep 1, 2017 at 2:43 PM, Rob Clark <robdclark at gmail.com> wrote:
> On Fri, Sep 1, 2017 at 3:13 AM, Laurent Pinchart
> <laurent.pinchart at ideasonboard.com> wrote:
>> Hi Nicolas,
>>
>> On Thursday, 31 August 2017 19:12:58 EEST Nicolas Dufresne wrote:
>>> Le jeudi 31 août 2017 à 17:28 +0300, Laurent Pinchart a écrit :
>>> >> e.g. if I have two devices which support MODIFIER_FOO, I could attempt
>>> >> to share a buffer between them which uses MODIFIER_FOO without
>>> >> necessarily knowing exactly what it is/does.
>>> >
>>> > Userspace could certainly set modifiers blindly, but the point of
>>> > modifiers is to generate side effects benefitial to the use case at hand
>>> > (for instance by optimizing the memory access pattern). To use them
>>> > meaningfully userspace would need to have at least an idea of the side
>>> > effects they generate.
>>>
>>> Generic userspace will basically pick some random combination.
>>
>> In that case userspace could set no modifier at all by default (except in the
>> case where unmodified formats are not supported by the hardware, but I don't
>> expect that to be the most common case).
>>
>>> To allow generically picking the optimal configuration we could indeed rely
>>> on the application knowledge, but we could also enhance the spec so that
>>> the order in the enumeration becomes meaningful.
>>
>> I'm not sure how far we should go. I could imagine a system where the API
>> would report capabilities for modifiers (e.g. this modifier lowers the
>> bandwidth, this one enhances the quality, ...), but going in that direction,
>> where do we stop ? In practice I expect userspace to know some information
>> about the hardware, so I'd rather avoid over-engineering the API.
>>
>
> I think in the (hopefully not too) long term, something like
> https://github.com/cubanismo/allocator/ is the way forward.  That
> doesn't quite solve how v4l2 kernel part sorts out w/ corresponding
> userspace .so what is preferable, but at least that is
> compartmentalized to v4l2.. on the gl/vk side of things there will ofc
> be a hardware specific userspace part that knows what it prefers.  For
> v4l2, it probably makes sense to sort out what the userspace level API
> is and work backwards from there, rather than risk trying to design a
> kernel uapi that might turn out to be the wrong thing.

I thought for kms the plan is to make the ordering meaningful, because
it doesn't necessarily match the gl/vk one. E.g. on intel gl would
prefer Y compressed, Y, X, untiled. Whereas display would be Y
compressed, X (much easier to scan out, in many cases allows more
planes to be used), Y (is necessary for 90° rotation), untiled. So if
drm_hwc really wants to use all the planes, it could prioritize the
display over rendering and request X instead of Y tiled.

I think the same would go for v4l.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch