[PATCH v2 0/3] Experimental freesync video mode optimization

[Christian König]

Am 11.12.20 um 13:20 schrieb Pekka Paalanen:
> On Fri, 11 Dec 2020 11:28:36 +0100
> Christian König <ckoenig.leichtzumerken at gmail.com> wrote:
>
>> Am 11.12.20 um 10:55 schrieb Pekka Paalanen:
>>> On Fri, 11 Dec 2020 09:56:07 +0530
>>> Shashank Sharma <shashank.sharma at amd.com> wrote:
>>>   
>>>> Hello Simon,
>>>>
>>>> Hope you are doing well,
>>>>
>>>> I was helping out Aurabindo and the team with the design, so I have
>>>> taken the liberty of adding some comments on behalf of the team,
>>>> Inline.
>>>>
>>>> On 11/12/20 3:31 am, Simon Ser wrote:
>>>>> Hi,
>>>>>
>>>>> (CC dri-devel, Pekka and Martin who might be interested in this as
>>>>> well.)
>>> Thanks for the Cc! This is very interesting to me, and because it was
>>> not Cc'd to dri-devel@ originally, I would have missed this otherwise.
>>>   
>>>>> On Thursday, December 10th, 2020 at 7:48 PM, Aurabindo Pillai <aurabindo.pillai at amd.com> wrote:
>>>>>      
>>>>>> This patchset enables freesync video mode usecase where the userspace
>>>>>> can request a freesync compatible video mode such that switching to this
>>>>>> mode does not trigger blanking.
>>>>>>
>>>>>> This feature is guarded by a module parameter which is disabled by
>>>>>> default. Enabling this paramters adds additional modes to the driver
>>>>>> modelist, and also enables the optimization to skip modeset when using
>>>>>> one of these modes.
>>>>> Thanks for working on this, it's an interesting feature! However I'd like to
>>>>> take some time to think about the user-space API for this.
>>>>>
>>>>> As I understand it, some new synthetic modes are added, and user-space can
>>>>> perform a test-only atomic *without* ALLOW_MODESET to figure out whether it can
>>>>> switch to a mode without blanking the screen.
>>>> The implementation is in those lines, but a bit different. The idea
>>>> is to:
>>>>
>>>> - check if the monitor supports VRR,
>>>>
>>>> - If it does, add some new modes which are in the VRR tolerance
>>>> range, as new video modes in the list (with driver flag).
>>>>
>>>> - when you get modeset on any of these modes, skip the full modeset,
>>>> and just adjust the front_porch timing
>>>>
>>>> so they are not test-only as such, for any user-space these modes
>>>> will be as real as any other probed modes of the list.
>>> But is it worth to allow a modeset to be glitch-free if the userspace
>>> does not know they are glitch-free? I think if this is going in, it
>>> would be really useful to give the guarantees to userspace explicitly,
>>> and not leave this feature at an "accidentally no glitch sometimes"
>>> level.
>>>
>>>
>>> I have been expecting and hoping for the ability to change video mode
>>> timings without a modeset ever since I learnt that VRR is about
>>> front-porch adjustment, quite a while ago.
>>>
>>> This is how I envision userspace making use of it:
>>>
>>> Let us have a Wayland compositor, which uses fixed-frequency video
>>> modes, because it wants predictable vblank cycles. IOW, it will not
>>> enable VRR as such.
>> Well in general please keep in mind that this is just a short term
>> solution for X11 applications.
> I guess someone should have mentioned that. :-)
>
> Do we really want to add more Xorg-only features in the kernel?

Well, my preferred solution was to add the mode in userspace instead :)

> It feels like "it's a short term solution for X11" could be almost used
> as an excuse to avoid proper uAPI design process. However, with uAPI
> there is no "short term". Once it's in, it's there for decades. So why
> does it matter if you design it for Xorg foremost? Are others forbidden
> to make use of it? Or do you deliberately want to design it so that
> it's not generally useful and it will indeed end up being a short term
> feature? Planned obsolescence from the start?

Yes exactly. We need something which works for now and can be removed 
again later on when we have a better solution. Adding some extra modes 
is not considered UAPI since both displays and drivers are doing this 
for a couple of different purposes already.

Another main reason is that this approach works with existing media 
players like mpv and kodi without changing them because we do pretty 
much the same thing in the kernel which TVs do in their EDID.

E.g. when starting to play a video kodi will automatically try to switch 
to a mode which has the same fps as the video.

>
>> For things like Wayland we probably want to approach this from a
>> completely different vector.
>>
>>> When the Wayland compositor starts, it will choose *some* video mode
>>> for an output. It may or may not be what a KMS driver calls "preferred
>>> mode", because it depends on things like user preferences. The
>>> compositor makes the initial modeset to this mode.
>> I think the general idea we settled on is that we specify an earliest
>> display time for each frame and give feedback to the application when a
>> frame was actually displayed.
> That is indeed something completely different, and feels like it would
> be several years in the future, while the proposed scheme or the
> min/max properties could be done fairly quickly. But I'm not in a hurry.

X11 already supports that with the DRI3 extension. Also see VDPAU 
present and the Vulkan extension for reference application API 
implementations, so that stuff is already present.

It's just not wired up correctly with KMS and we don't have anything 
similar in Wayland as far as I know.

> Setting the earliest display time for a flip does not fully cover what
> video mode timing adjustment or min/max frame time or refresh rate
> property would offer: prediction of when the next flip can happen.
> Choosing display times requires knowing the possible display times, so
> something more is needed. The min/max properties would fit in.

Well you need to communicate to the application what the minimum and 
maximum refresh rates are by some extension of the mode description.

Regards,
Christian.


>
>> This approach should also be able to handle multiple applications with
>> different refresh rates. E.g. just think of a video playback with 25 and
>> another one with 30 Hz in two windows when the max refresh rate is
>> something like 120Hz.
> But that's just an extension to what I already described and completely
> possible with the proposed and the min/max approaches.
>
>
> Thanks,
> pq