DRM Format Modifiers in v4l2

[Hans Verkuil]

On 30/08/17 11:36, Brian Starkey wrote:
> On Wed, Aug 30, 2017 at 10:10:01AM +0200, Hans Verkuil wrote:
>> On 30/08/17 09:50, Daniel Vetter wrote:
>>> On Tue, Aug 29, 2017 at 10:47:01AM +0100, Brian Starkey wrote:
>>>> The fact is, adding special formats for each combination is
>>>> unmanageable - we're talking dozens in the case of our hardware.
>>>
>>> Hm right, we can just remap the special combos to the drm-fourcc +
>>> modifier style. Bonus point if v4l does that in the core so not everyone
>>> has to reinvent that wheel :-)
>>
>> Probably not something we'll do: there are I believe only two drivers that
>> are affected (exynos & mediatek), so they can do that in their driver.
>>
>> Question: how many modifiers will typically apply to a format? I ask
>> because I realized that V4L2 could use VIDIOC_ENUMFMT to make the link
>> between a fourcc and modifiers:
>>
>> https://hverkuil.home.xs4all.nl/spec/uapi/v4l/vidioc-enum-fmt.html
>>
>> The __u32 reserved[4] array can be used to provide a bitmask to modifier
>> indices (for the integer menu control). It's similar to what drm does,
>> except instead of modifiers mapping to fourccs it is the other way around.
>>
>> This would avoid having to change the modifiers control whenever a new
>> format is set and it makes it easy to enumerate all combinations.
>>
>> But this only works if the total number of modifiers used by a single driver
>> is expected to remain small (let's say no more than 64).
> 
> In our current (yet to be submitted) description, we've got around a
> dozen modifiers for any one format to describe our compression
> variants. We have a lot of on/off toggles which leads to combinatorial
> expansion, so it can grow pretty quickly (though I am trying to limit
> the valid combinations as much as possible).
> 
> How about if the mask fills up then VIDIOC_ENUM_FMT can return another
> fmtdsc with the same FourCC and different modifier bitmask, where the
> second one's modifier bitmask is for the next "N" modifiers?

I was thinking along similar lines, but it could cause some problems with
the ABI since applications currently assume that no fourcc will appear
twice when enumerating formats. Admittedly, we never explicitly said in
the spec that that can't happen, but it is kind of expected.

There are ways around that, but if possible I'd like to avoid that.

In theory there are up to 128 bits available but I can't help thinking
that if you create more than, say, 64 modifiers for a HW platform you
have a big mess anyway.

If I am wrong, then I need to know because then I can prepare for it
(or whoever is going to actually implement this...)

If the number of modifiers is expected to be limited then making 64 bits
available would be good enough, at least for now.

BTW, is a modifier always optional? I.e. for all fourccs, is the unmodified
format always available? Or are there fourccs that require the use of a
modifier?

Regards,

	Hans