[RFC v2] drm/kms: control display brightness through drm_connector properties

[Hans de Goede]

Hi,

On 9/30/22 18:17, Sebastian Wick wrote:
> On Fri, Sep 30, 2022 at 5:27 PM Pekka Paalanen <ppaalanen at gmail.com> wrote:
>>
>> On Fri, 30 Sep 2022 17:44:17 +0300
>> Ville Syrjälä <ville.syrjala at linux.intel.com> wrote:
>>
>>> On Fri, Sep 30, 2022 at 04:20:29PM +0200, Sebastian Wick wrote:
>>>> On Fri, Sep 30, 2022 at 9:40 AM Pekka Paalanen <ppaalanen at gmail.com> wrote:
>>>>>
>>>>> On Thu, 29 Sep 2022 20:06:50 +0200
>>>>> Sebastian Wick <sebastian.wick at redhat.com> wrote:
>>>>>
>>>>>> If it is supposed to be a non-linear luminance curve, which one is it?
>>>>>> It would be much clearer if user space can control linear luminance
>>>>>> and use whatever definition of perceived brightness it wants. The
>>>>>> obvious downside of it is that it requires bits to encode changes that
>>>>>> users can't perceive. What about backlights which only have a few
>>>>>> predefined luminance levels? How would user space differentiate
>>>>>> between the continuous and discrete backlight? What about
>>>>>> self-emitting displays? They usually increase the dynamic range when
>>>>>> they increase in brightness because the black level doesn't rise. They
>>>>>> also probably employ some tonemapping to adjust for that. What about
>>>>>> the range of the backlight? What about the absolute luminance of the
>>>>>> backlight? I want to know about that all in user space.
>>>>>>
>>>>>> I understand that most of the time the kernel doesn't have answers to
>>>>>> those questions right now but the API should account for all of that.
>>>>>
>>>>> Hi,
>>>>>
>>>>> if the API accounts for all that, and the kernel doesn't know, then how
>>>>> can the API not lie? If the API sometimes lies, how could we ever trust
>>>>> it at all?
>>>>
>>>> Make it possible for the API to say "I don't know". I'd much rather
>>>> have an API tell me explicitly what it does and doesn't know instead
>>>> of having to guess what data I can actually rely on.
>>>>
>>>> For example if the kernel knows the luminance is linear on one display
>>>> and doesn't know anything about the other display and it exposes them
>>>> both in the same way I can not possibly write any code which relies on
>>>> exact control over the luminance for either display.
>>>>
>>>>>
>>>>> Personally I have the feeling that if we can even get to the level of
>>>>> "each step in the value is a more or less perceivable change", that
>>>>> would be good enough. Think of UI, e.g. hotkeys to change brightness.
>>>>> You'd expect almost every press to change it a bit.
>>>>
>>>> The nice thing is that you can have that even if you have no further
>>>> information about the brightness control and it might be good enough
>>>> for some use cases but it isn't for others.
>>>>
>>>>> If an end user wants defined and controlled luminance, I'd suggest they
>>>>> need to profile (physically measure) the response of the display at
>>>>> hand. This is no different from color profiling displays, but you need
>>>>> a measurement device that produces absolute measurements if absolute
>>>>> control is what they want.
>>>>
>>>> If that's the kind of user experience you're after, good for you. I
>>>> certainly want things to work out of the box which makes this just a
>>>> big no-go.
>>>
>>> I think if we have the information to make the default behaviour
>>> better then we should do that. Ie. if the firmaware gives us a
>>> table to remap the values for a more linear response we should
>>> make use of that by default.
>>
>> But that's only like 20% of what Sebastian is asking for.
>>
>> What's "linear"? Radiometric or perceptual?
>>
>> Radiometric linear control would make a terrible UX, so if the control
>> is radiometric, userspace needs to remap it. That might be a good
>> thing, but it's also complicated, because the relationship between
>> brightness and luminance is somewhere between a power curve and
>> exponential curve. You need to make sure that the userspace remapping
>> works for different backlights with different luminance ranges. That's
>> not obvious to me.
>>
>>> We can of course provide a way for the user to plug in their own
>>> actually measured data later. But IMO that doesn't even have to
>>> happen in the initial implementation. Just need to avoid painting
>>> ourselves totally in the corner in a way that would prevent later
>>> additions like that.
>>
>> For userspace delivering its own curve, you need to define the units.
>> Absolute or relative? Radiometric or perceptual? Otherwise the
>> resulting control is not portable between window systems.
>>
>>> I just hate the current limbo where we're somehow too afraid to
>>> change the current behaviour to do the remapping by default.
>>> I see no upsides in the current behaviour of just blindly
>>> exposing the raw hardware register values more or less. They
>>> mean absolutely nothing to any user.
>>
>> I never argued like that.
>>
>> I'm saying that what looks realistic to me is somewhere *between*
>> status quo and what Sebastian is asking for. Whatever you mean by "linear
>> remapping" is probably a realistic goal, because you know you have some
>> hardware/firmware delivering that information already.
>>
>> OTOH, designing UAPI for information that exists only in our dreams
>> is... well.
> 
> I also didn't say that we should design an UAPI for what doesn't
> exist, just that we should design the UAPI we actually need in a way
> that when we get more information we can properly expose that. So if
> the UAPI exposes anything other than the luminance (e.g. "each step is
> a perceptual step in brightness", "linear brightness", ..) we have to
> put some human perception model into the kernel which is ridiculous
> and it makes it impossible to expose luminance to user space if the
> kernel has that information.
> 
> It's really easy to paint yourself into a corner here.

The problem is we are already in that corner and this is not something
which we can fix in the kernel.

I agree that we don't want to put a "human perception model" model
into the kernel. But at the same time the scale of the brightness
control methods already is widely unknown.

Simplifying things greatly there are 2 groups of brightness control
methods:

1. Some firmware interface
2. Directly programming a raw PWM output

For 1. we have no clue of the scale it might be linear electrical
power send to the backlight. But it often already has some perceived
brightness curve applied with the exact curve unknown.

For 2. The PWM might directly turn the LEDs on/off rapidly, making
the duty-cycle a 1:1 linear map of the amount of Watts consumed
by the backlight. But on some (a lot?) have the raw PWM output is
used as input to a LED driver IC which is basically a PWM controlled
current-source. There might be a 1:1 mapping here between the PWM
input and the amount of Ampere coming out of the current-source.
OTOH these backlight drive ICs often have the option to load
some PWM -> Amperage curve in them to do something like
perceived brightness again. And the amount of nits coming out
of the LEDs is also not a linear function of the Amperage going
into the LEDs.

Sebastian, I understand that you would really like a control
where you can just say, amount of nits coming form the screen
is 100 / 200 / 400 (when screen is fully white). But we don't
even know how many nits the backlight (or OLEDs) will send
at full brightness settings, we don't know what if any curve
is being applied by the firmware and/or by some backlight driver
IC and even if knew all that the amount of nits will change
over time due to wear and tear.

So I understand that in an ideal world you would want all this
info for HDR support, but we simply do not have it.

So for now it is important to at least move brightness control
into KMS using a simple uAPI where the only thing we can really
say about the scale is higher values result in a higher brightness.

And then once that is in place we can look at trying to export
more information; in so far as we have that information.

One example mentioned here is that laptops with Intel integrated
graphics may have some "perceived brightness" mapping table
in their Video BIOS Tables (VBT) it would be interesting to use
this and to export the curve coming from that to userspace as
extra information (including allow userspace to write it).
Since in this case (unlike many other cases) at least this
curve is done in the kernel I guess we can then also add a boolean
property to disable the curve (making it linear) in case that
is helpful for HDR scenarios.

But I'm afraid that for now we really need to play it by ear
here since so much of what you want is hidden from the kernel
and will differ greatly from one model laptop to the next.

Regards,

Hans