RFC for a render API to support adaptive sync and VRR

[Michel Dänzer]

On 2018-04-11 08:57 AM, Nicolai Hähnle wrote:
> On 10.04.2018 23:45, Cyr, Aric wrote:
>>>>>>>>> For video games we have a similar situation where a frame is
>>>>>>>>> rendered
>>>>>>>>> for a certain world time and in the ideal case we would actually
>>>>>>>>> display the frame at this world time.
>>>>>>>>
>>>>>>>> That seems like it would be a poorly written game that flips like
>>>>>>>> that, unless they are explicitly trying to throttle the
>>>>>>>> framerate for
>>>>>>>> some reason.  When a game presents a completed frame, they’d like
>>>>>>>> that to happen as soon as possible.
>>>>>>>
>>>>>>> What you're describing is what most games have been doing
>>>>>>> traditionally.
>>>>>>> Croteam's research shows that this results in micro-stuttering,
>>>>>>> because
>>>>>>> frames may be presented too early. To avoid that, they want to
>>>>>>> explicitly time each presentation as described by Christian.
>>>>>>
>>>>>> Yes, I agree completely.  However that's only truly relevant for
>>>>>> fixed
>>>>>> refreshed rate displays.
>>>>>
>>>>> No, it also affects variable refresh; possibly even more in some
>>>>> cases,
>>>>> because the presentation time is less predictable.
>>>>
>>>> Yes, and that's why you don't want to do it when you have variable
>>>> refresh.  The hardware in the monitor and GPU will do it for you,
>>> so why bother?
>>>
>>> I think Michel's point is that the monitor and GPU hardware *cannot*
>>> really do this, because there's synchronization with audio to take into
>>> account, which the GPU or monitor don't know about.
>>
>> How does it work fine today given that all kernel seems to know is
>> 'current' or 'current+1' vsyncs.
>> Presumably the applications somehow schedule all this just fine.
>> If this works without variable refresh for 60Hz, will it not work for
>> a fixed-rate "48Hz" monitor (assuming a 24Hz video)?
> 
> You're right. I guess a better way to state the point is that it
> *doesn't* really work today with fixed refresh, but if we're going to
> introduce a new API, then why not do so in a way that can fix these
> additional problems as well?

Exactly. With a fixed frame duration, we'll still have fundamentally the
same issues as we currently do without variable refresh, not making use
of the full potential of variable refresh.


> Say you have a multi-GPU system, and each GPU has multiple displays
> attached, and a single application is driving them all. The application
> queues flips for all displays with the same target_present_time_ns
> attribute. Starting at some time T, the application simply asks for the
> same present time T + i * 16666667 (or whatever) for frame i from all
> displays.

BTW, this is an interesting side point I've wanted to make: Any
applications / use cases which really do want a fixed refresh rate can
trivially do it with time-based presentation like this.


> Of course it's to be expected that some (or all) of the displays will
> not be able to hit the target time on the first bunch of flips due to
> hardware limitations, but as long as the range of supported frame times
> is wide enough, I'd expect all of them to drift towards presenting at
> the correct time eventually, even across multiple GPUs, with this simple
> scheme.
> 
> Why would that not work to sync up all displays almost perfectly?

Seconded.


> How about what I wrote in an earlier mail of having attributes:
> 
> - target_present_time_ns
> - hint_frame_time_ns (optional)
> 
> ... and if a video player set both, the driver could still do the
> optimizations you've explained?

FWIW, I don't think a property would be a good mechanism for the target
presentation time.

At least with VDPAU, video players are already explicitly specifying the
target presentation time, so no changes should be required at that
level. Don't know about other video APIs.

The X11 Present extension protocol is also prepared for specifying the
target presentation time already, the support for it just needs to be
implemented.


-- 
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer