[PATCH v2 2/3] drm: Add variable refresh property to DRM CRTC

Mon Oct 15 16:02:14 UTC 2018

On 10/15/2018 09:57 AM, Pekka Paalanen wrote:
> On Fri, 12 Oct 2018 08:58:23 -0400
> "Kazlauskas, Nicholas" <nicholas.kazlauskas at amd.com> wrote:
> 
>> On 10/12/2018 07:20 AM, Koenig, Christian wrote:
>>> Am 12.10.2018 um 11:21 schrieb Pekka Paalanen:
>>>> On Fri, 12 Oct 2018 07:35:57 +0000
>>>> "Koenig, Christian" <Christian.Koenig at amd.com> wrote:
>>>>   
>>>>> Am 12.10.2018 um 09:23 schrieb Pekka Paalanen:
>>>>>> On Wed, 10 Oct 2018 09:35:50 -0400
>>>>>> "Kazlauskas, Nicholas" <nicholas.kazlauskas at amd.com> wrote:
>>>>>>       
>>>>>>> The patches I've put out target this use case mostly because of the
>>>>>>> benefit for a relatively simple interface. VRR can and has been used in
>>>>>>> more ways that this, however.
>>>>>>>
>>>>>>> An example usecase that differs from this would actually be video
>>>>>>> playback. The monitor's refresh rate likely doesn't align with the video
>>>>>>> content rate. An API that exposes direct control over VRR (like the
>>>>>>> range, refresh duration, presentation timestamp) could allow the
>>>>>>> application to specify the content rate directly to deliver a smoother
>>>>>>> playback experience. For example, if you had a 24fps video and the VRR
>>>>>>> range was 40-60Hz you could target 48Hz via some API here.
>>>>>> The way that has been discussed to be implemented is that DRM page flips
>>>>>> would carry a target timestamp, which the driver will then meet as good
>>>>>> as it can. It is better to define an absolute target timestamp than a
>>>>>> frequency, because a timestamp can be used to synchronize with audio
>>>>>> and more. Mario Kleiner can tell you all about scientific use cases
>>>>>> that require accurate display timing, not just frequency.
>>>>> To summarize what information should be provided by the driver stack to
>>>>> make applications happy:
>>>>>
>>>>> 1. The minimum time a frame can be displayed, in other words the maximum
>>>>> frame rate.
>>>>> 2. The maximum time a frame can be displayed, in other words the minimum
>>>>> frame rate.
>>>>> 3. How much change of frame timing is allowed between frames to avoid
>>>>> luminescence flickering.
>>>>>
>>>>> Number 1 and 2 can also be used to signal the availability of VRR to
>>>>> applications, e.g. if they are identical we don't support VRR at all.
>>>> Hi Christian,
>>>>
>>>> "the maximum time a frame can be displayed" is perhaps not an
>>>> unambiguous definition. A frame can be shown indefinitely in any case.
>>>
>>> Good point. Please also note that I'm not an expert on the display stuff
>>> in general.
>>>
>>> My background comes more from implementing the VDPAU mediaplayer
>>> interface in mesa.
>>>
>>> So just throwing some ideas in here from the point of view of an
>>> userspace developer which wants to write a media player :)
> 
> Excellent!
> 
>>>> The CRTC will simply start scanning out the same frame again if there
>>>> is no new one. I understand what you want to say, but perhaps some
>>>> different words will be in order.
>>>>
>>>> I do wonder if applications really want to know the maximum acceptable
>>>> slew rate in timings... maybe that should be left for the drivers to
>>>> apply. What I'm thinking is that we have the page flip timestamp with
>>>> the page flip events to tell when the new FB became active. That
>>>> information could be extended with a time range on when the very next
>>>> flip could take place. Applications are already computing that
>>>> prediction from the flip timestamp and fixed refresh rate, but it might
>>>> be nice to give them the driver's opinion explicitly. Maybe the
>>>> tolerable slew rate is not a constant.
>>>
>>> Well it depends. I agree that the kernel should probably enforce the
>>> slew rate to avoid flickering for the user.
> 
> That's what I was hoping if the monitor hardware does not do that
> already, but now it sounds like it's not possible. Another failed
> assumption from my side.
> 
>>> But it might be beneficial for the application to know things like this.
> 
> Applications should know when they could likely flip, my question is
> how to tell them. Is the acceptable slew rate a constant for a video
> mode, or does it depend on the previous refresh interval.
> 
>>>
>>> What the application definitely needs to know is when a frame was
>>> actually displayed. E.g. application says I want to display this at
>>> point X and at some point later the kernel returns saying the frame was
>>> displayed at point N where in the ideal case N=X.
> 
> You already get this timestamp with the DRM page flip events. We also
> have Wayland and X11 extensions to get it to apps.
> 
>>>
>>> Additional to that let's please use 64bit values and nanoseconds for
>>> every value we use here, and NOT fps, line numbers or similar :)
> 
> Seconded!
> 
> I'd really favour absolute timestamps even.
> 
>>>
>>> Regards,
>>> Christian.
>>
>> Flickering will really depend on the panel itself. A wider ranger is
>> more likely to exhibit the issue, but factors like topology, pixel
>> density and other display technology can affect this.
>>
>> It can be hard for a driver to guess at all of this. For many panels
>> it'd be difficult to notice unless it's consistently jumping between the
>> min and max range.
> 
> Do you mean that there is no way to know and get that information from
> the monitor itself? Nothing in EDID that could even be used as a
> default and quirk the monitors that got it wrong?

The variable refresh range is exposed, but that's about it. There's 
certainly no simple algorithm you can feed monitor information into to 
determine how bad the luminance flickering is going to look to the user.

> 
>> Opening up an API that restricts how much the driver can change the
>> refresh rate in a VRR scenario seems a bit extreme, but there's probably
>> some applications that could benefit from this. I'd certainly want to
>> see the actual use case first, though. This still feels more like a
>> driver policy to me.
> 
> I don't think anyone suggested that, certainly I did not. The driver
> should get that information from the monitor hardware so that it can
> drive it correctly. If the hardware driver doesn't know, then how could
> the DRM core or userspace know any better? My whole premise was that the
> driver knows.

This was referring to the "How much change of frame timing is allowed 
between frames to avoid luminescence flickering" comment. I assumed that 
this implied restriction of the min/max VRR ranges from the driver.

> 
> Sounds like VRR hardware was designed not only for a consistent refresh
> rate but also temporary glitches being ok.
> 
> I suppose this will result in choosing a very pessimistic allowed slew
> rate in the driver that covers 95% of VRR monitors and handle the rest
> with quirks. That could still work.
> 
>> I agree with the nanosecond based timestamp API making the most logical
>> sense here from an API perspective. This does overlap a little bit with
>> the target vblank property that's already on the CRTC, perhaps.
> 
> KMS UABI already has a target vblank property defined, or are you
> talking about your CRTC hardware?
> 
> Target vblank counter value makes even less sense with VRR than it ever
> did with a fixed refresh rate. :-) >
>> The target vblank could be determined based on the timestamp. The driver
>> is likely to exceed the target presentation timestamp by a fair bit if
>> it was to just do this, however. VRR could be used in this case to get
>> closer to the timestamp. A naive implementation could iterate over every
>> rate in the range and take the one with the minimum difference, for
>> example.
> 
> Could you elaborate on that, who could be doing what exactly to achieve
> what?
> 
> 
> Thanks,
> pq
> 

These comments were mostly discussing about how a kernel driver might 
approach a target presentation timestamp.

A target presentation timestamp isn't related to VRR directly. Without 
VRR this could still be implemented by a kernel driver, but it would 
effectively be the client asking the driver to pick the right target 
vblank. There's already some support for this in DRM with the 
target_vblank CRTC property, but it isn't the easiest thing to use.

The driver can do much better at getting closer to the client's target 
presentation timestamp by adjusting the refresh rate accordingly on a 
VRR capable monitor.

Nicholas Kazlauskas