[PATCH] Add freesync ioctl interface

Tue Aug 9 10:44:36 UTC 2016

Am 09.08.2016 um 12:03 schrieb Michel Dänzer:
> On 09/08/16 06:31 PM, Christian König wrote:
>> Am 09.08.2016 um 10:27 schrieb Michel Dänzer:
>>> On 09/08/16 05:12 PM, Christian König wrote:
>>>> Am 09.08.2016 um 04:44 schrieb Michel Dänzer:
>>>>
>>>>> I was basically thinking out loud that doing this via different modes
>>>>> might be quite natural, *if* games allowed choosing a specific mode.
>>>>> But unfortunately they don't. For the video playback case, how do you
>>>>> envision the video player app communicating the refresh rate of the
>>>>> currently playing video to the kernel?
>>>> Again the kernel doesn't need to know the refresh rate. All the kernel
>>>> needs to know is when to do the page flip.
>>>>
>>>> So coming back to my example of a mode with 1920x1080 and 20-100Hz
>>>> refresh rate a classic modeline would then look something like this:
>>>>
>>>> Modeline "1920x1080_dynamic"  302.50  1920 2072 2280 2640  1080 1083
>>>> 1088 5735 -hsync +vsync
>>>>
>>>> Note the high vertical total scan lines. Those basically reduce the
>>>> refresh rate from 100Hz (which this mode normally would have) down to
>>>> only 20Hz.
>>>>
>>>> Now what userspace does on each page flip is specifying when this flip
>>>> should happen, e.g. when the frame should be started to be scanned out.
>>>> We can either specify this as frame counter + vertical line of the
>>>> previous frame or as something like CLOCK_MONOTONIC (I think I would
>>>> prefer CLOCK_MONOTONIC, but that's not a strong opinion).
>>>>
>>>> In other words you put the whole concept upside down. It's no longer the
>>>> kernel which tells userspace when a vblank happened, but rather
>>>> userspace tells the kernel when it should happen (together with the
>>>> frame that should be displayed then).
>>> I guess that could work. Do video players set the VDPAU presentation
>>> time accurately enough for this?
>> Yes, of course. We actually get a precise time stamp from the
>> application and need to calculate on which vblank to display it from that.
>>
>>> This would require extensive changes across the stack though, when more
>>> or less the same result could be achieved by just letting the kernel
>>> know what the current refresh rate is supposed to be, e.g. via output
>>> properties.
>> The problem is that you don't have a refresh rate any more.
> Maybe not below the video decoding API level, but the video player app
> certainly knows the refresh rate?

Sure, but as I said that isn't a constant refresh rate any more.

E.g. the refresh rate from the video header doesn't need to be a 
multiple of the time a vertical line takes to display.

This results in sightly different display times for each frame. So you 
don't have a constant frame rate any more but rather alternate between 
two (or maybe more) different display times for each frame.

I mean we could of course adjust the frame rate the kernel should 
display with each page flip as well, but that certainly isn't how the 
hardware is working now.

>> Mostly the same applies for games as well, e.g. when you render a frame
>> you usually render it for a certain timestamp.
> I can only find the EGL_ANDROID_presentation_time extension providing
> exactly this functionality. GLX_NV_present_video looks like it covers
> this as well, but it also includes a bunch of other stuff.
>
> I think 0 is a good first approximation of the number of applications
> used by an average Linux desktop user which make use of either of those
> extensions.

Yeah, I agree that we don't have proper support for this on Linux 
desktop at the moment. But this is a chicken and egg problem.

The comment that games render a frame for a certain timestamp was more 
on how games usually work internally.

>> Additional to that are you sure it is such a hassle to implement this? I
>> mean let us sum up what we need:
>> 1. A representation for the new mode attributes, e.g. minimum and
>> maximum vertical refresh rate.
>>
>> This is needed anyway to proper communicate the capabilities of the
>> display device to userspace.
>>
>> 2. An extension to the page flip IOCTL to specify when exactly a flip
>> should happen.
>>
>> As far as I can see that is what your patchset already did. The only
>> difference is that you wanted to specify a certain vertical blank when
>> the flip would happen while I would say we should use a monotonic
>> timestamp (64bit ns since boot) for this.
> Right, the patch series you're referring to isn't directly related to this.
>
>
> 3. Expose this new functionality via the X11 Present extension and/or a
> corresponding Wayland extension.
I unfortunately haven't pushed on this, but I already noted to Keith on 
an earlier version of the present extension that we should use a 
timestamps instead of vertical blank counters.

But double checking the present extension, we indeed already have 
support for PresentOptionUST in it. So this should already be there.

> 4. Make use of 3. in the Gallium video state tracker code.
>
> Now we're done for the video playback case, phew!
>
>
> 5. Make use of 3. to implement EGL_ANDROID_presentation_time and/or
> GLX_NV_present_video.
>
> 6. Make game engines use 5.
>
> (A game engine can know when a frame will be ready for presentation when
> it starts rendering it, so I'm not sure this functionality will be very
> useful for games)
>
>
>>> Also, this doesn't address the case of running (existing) games with
>>> variable refresh rate.
>> Sure it does. For the current stack without any change a freesync
>> capable display device just looks like a normal monitor with a high
>> vertical refresh rate.
> You're saying current userspace would just see the maximum vertical
> refresh rate as the refresh rate?

I think so. I mean the minimum is usually rather low, e.g. between 20 
and 30Hz.

>
>
>> When we add freesync support we just extend vblank_mode with a new enum
>> to enable it optionally for existing applications.
> "Just"... this will only be possible after the first 3 steps above.

Checking the present extension again we already got PresentOptionAsync 
in there as well.

So the whole thing boils down implementing a new vblank_mode enume which 
sets PresentOptionAsync and then doing the right thing on the X side 
with those information.

I'm going to give the UST option a try with VDPAU, let's see how well 
that works and if it is implemented correctly or not.

Regards,
Christian.