RFC: page-flip with damage?

[Thomas Hellstrom]

On 09/26/2017 10:18 AM, Daniel Vetter wrote:
> On Sun, Sep 24, 2017 at 07:41:45PM +0200, Thomas Hellstrom wrote:
>> Hi, list!
>>
>> Page flips, while efficient on real hardware, aren't that efficient in other
>> situations, like for virtual devices with local, or even worse, remote
>> desktops.
>> We might ending up forwarding or encoding a couple of full frames worth of
>> data instead of a small region at a cursor blink.
>>
>> Now there is this extension EGL_KHR_swap_buffers_with_damage, and
>> gnome-shell/wayland on KMS also has a damage region that it forwards all the
>> way down to the function where page-flip is called.
>>
>> So I'd like to start looking at page-flips with damage, meaning that the
>> damage is an optional hint to the device about what part of the contents is
>> actually updated. What would be the best way to implement this? I figure
>> this can be done within the atomic context with a region attached to the
>> plane state? Would we want to follow the EGL extension and forward an array
>> of rects or for simplicity use a single bounding box? Both these options
>> would be a great win.
> So my rough plan for all this was:
>
> - Add damage to drm_crtc_state, in screen coordinates. I think this is the
>    most natural place for this, since it's what PSR/manual upload DSI want.
>    It should also fit well for udl and tinydrm.  Virtual drivers like
>    vmwgfx might need helpers to wrap this back to framebuffer rectangles,
>    but that seems the odd case out - the framebuffer-based approach in the
>    DIRTY_IOCTL forces most drivers to do a fancy lookup from fb to the
>    crtc.
>
>    Per-plane dirty rectangle seems to be an awkward in-betwen state, with
>    all the confusion about whether it's pre/post scaled and how to best
>    combine them. And then someone changes the background color of the crtc
>    (or something like that), what happens then? I think pushing all that
>    onto userspace is best, it can always ask for a complete flip if it's
>    unclear whether it damaged the entire screen or not.

Actually, after looking into this a bit in the context of remoting, I 
think I will have to disagree.

The most natural place for damage appears to be the drm_plane state, in 
plane fb coordinates.
The reason for this is that devices with hardware planes will want to 
have the coordinates in this way. Think cursor, Video overlay. Currently 
with vmwgfx and atomic we need to send the cursor image down the device 
pipeline on each cursor move which is pretty nasty.
Same thing holds for video overlay if we decide to move it to atomic. 
The overlay engine wants to know what part of the source image
has changed.
Damage is triggered by a content change and the change lives in the 
framebuffer, and this is easier on user-space as well.

Now for software plane compositing, you might be hitting the problems 
you describe above.
But if you're doing software plane compositing in your drm driver (which 
you probably aren't),
you'd have enough information anyway to handle both the odd case of 
background color change and any scaling if present (god forbid).

Thanks,
Thomas