RFC for a render API to support adaptive sync and VRR

[Harry Wentland]

On 2018-04-10 12:37 PM, Nicolai Hähnle wrote:
> On 10.04.2018 18:26, Cyr, Aric wrote:
>> That presentation time doesn’t need to come to kernel as such and actually is fine as-is completely decoupled from adaptive sync.  As long as the video player provides the new target_frame_duration_ns on the flip, then the driver/HW will target the correct refresh rate to match the source content.  This simply means that more often than not the video presents will  align very close to the monitor’s refresh rate, resulting in a smooth video experience.  For example, if you have 24Hz content, and an adaptive sync monitor with a range of 40-60Hz, once the target_frame_duration_ns is provided, driver can configure the monitor to a fixed refresh rate of 48Hz causing all video presents to be frame-doubled in hardware without further application intervention.
> 
> What about multi-monitor displays, where you want to play an animation that spans multiple monitors. You really want all monitors to flip at the same time.
> 

Syncing two monitors is what we currently do with our timing sync feature where we drive two monitors from the same clock source if they use the same timing. That, along with VSync, guarantees all monitors flip at the same time. I'm not sure if it works with adaptive sync.

Are you suggesting to use adaptive sync to do an in-SW sync of multiple displays?

> I understand where you're coming from, but the perspective of refusing a target presentation time is a rather selfish one of "we're the display, we're the most important, everybody else has to adjust to us" (e.g. to get perfect sync between video and audio). I admit I'm phrasing it in a bit of an extreme way, but perhaps this phrasing helps to see why that's just not a very good attitude to have.
> 

I really dislike arguing on an emotional basis and would rather not use words such as "selfish" in this discussion. I believe all of us want to come to the best possible solution based on technical merit.

> All devices (whether video or audio or whatever) should be able to receive a target presentation time.
> 

I'm not sure I understand the full extent of the problem as I'm not really familiar with how this is currently done, but isn't the problem the same without variable refresh rates (or targeted refresh rates)? A Video API would still have to somehow synchronize audio and video to 60Hz on most monitors today. What would change if we gave user mode the ability to suggest we flip at video frame rates (24/48Hz)?

Harry

> If the application can make your life a bit easier by providing the targetted refresh rate as additional *hint-only* parameter (like in your 24 Hz --> 48 Hz doubling example), then maybe we should indeed consider that.
> 
> Cheers,
> Nicolai
> 
> 
>>
>>
>> For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.
>>
>> That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason.  When a game presents a completed frame, they’d like that to happen as soon as possible.  This is why non-VSYNC modes of flipping exist and many games leverage this.  Adaptive sync gives you the lower latency of immediate flips without the tearing imposed by using non-VSYNC flipping.
>>
>>
>> I mean we have the guys from Valve on this mailing list so I think we should just get the feedback from them and see what they prefer.
>>
>> We have thousands of Steam games on other OSes that work great already, but we’d certainly be interested in any additional feedback.  My guess is they prefer to “do nothing” and let driver/HW manage it, otherwise you exempt all existing games from supporting adaptive sync without a rewrite or update.
>>
>>
>> Regards,
>> Christian.
>>
>>
>>     -Aric
>>
>