RFC: page-flip with damage?

[Daniel Vetter]

On Mon, Nov 27, 2017 at 09:03:22AM +0100, Thomas Hellstrom wrote:
> On 09/26/2017 10:18 AM, Daniel Vetter wrote:
> > On Sun, Sep 24, 2017 at 07:41:45PM +0200, Thomas Hellstrom wrote:
> > > Hi, list!
> > > 
> > > Page flips, while efficient on real hardware, aren't that efficient in other
> > > situations, like for virtual devices with local, or even worse, remote
> > > desktops.
> > > We might ending up forwarding or encoding a couple of full frames worth of
> > > data instead of a small region at a cursor blink.
> > > 
> > > Now there is this extension EGL_KHR_swap_buffers_with_damage, and
> > > gnome-shell/wayland on KMS also has a damage region that it forwards all the
> > > way down to the function where page-flip is called.
> > > 
> > > So I'd like to start looking at page-flips with damage, meaning that the
> > > damage is an optional hint to the device about what part of the contents is
> > > actually updated. What would be the best way to implement this? I figure
> > > this can be done within the atomic context with a region attached to the
> > > plane state? Would we want to follow the EGL extension and forward an array
> > > of rects or for simplicity use a single bounding box? Both these options
> > > would be a great win.
> > So my rough plan for all this was:
> > 
> > - Add damage to drm_crtc_state, in screen coordinates. I think this is the
> >    most natural place for this, since it's what PSR/manual upload DSI want.
> >    It should also fit well for udl and tinydrm.  Virtual drivers like
> >    vmwgfx might need helpers to wrap this back to framebuffer rectangles,
> >    but that seems the odd case out - the framebuffer-based approach in the
> >    DIRTY_IOCTL forces most drivers to do a fancy lookup from fb to the
> >    crtc.
> > 
> >    Per-plane dirty rectangle seems to be an awkward in-betwen state, with
> >    all the confusion about whether it's pre/post scaled and how to best
> >    combine them. And then someone changes the background color of the crtc
> >    (or something like that), what happens then? I think pushing all that
> >    onto userspace is best, it can always ask for a complete flip if it's
> >    unclear whether it damaged the entire screen or not.
> 
> Actually, after looking into this a bit in the context of remoting, I think
> I will have to disagree.
> 
> The most natural place for damage appears to be the drm_plane state, in
> plane fb coordinates.
> The reason for this is that devices with hardware planes will want to have
> the coordinates in this way. Think cursor, Video overlay. Currently with
> vmwgfx and atomic we need to send the cursor image down the device pipeline
> on each cursor move which is pretty nasty.
> Same thing holds for video overlay if we decide to move it to atomic. The
> overlay engine wants to know what part of the source image
> has changed.
> Damage is triggered by a content change and the change lives in the
> framebuffer, and this is easier on user-space as well.
> 
> Now for software plane compositing, you might be hitting the problems you
> describe above.
> But if you're doing software plane compositing in your drm driver (which you
> probably aren't),
> you'd have enough information anyway to handle both the odd case of
> background color change and any scaling if present (god forbid).

The one thing I'm somewhat worried about with per-plane damage rects is
correctly adding them up into an overall crtc rect for hw that needs it
(that's edp and manual upload dsi). There's some fixed point math involved
first to get from plane src to crtc dst coords, and then there's the fun
of figuring out which properties affect the entire screen or entire plane
(background, gamma tables, ...).

But I guess we can do that in a helper and share that code with all
drivers that need it.

I think we have 2 different kinds of drivers which would benefit from
this:

- drivers that composite on the "display"/remotely (like vmwgfx, or maybe
  also spice): Want damage in plane source coordinates. I think we could
  also group udl and spi/i2c-connected screens into this, since you can do
  the same buffer upload dance. Those drivers also might benefit from a
  list of rectangles (assuming the rectangles aren't too small). At least
  e.g. udl has a hw cursor which is also composited remotely.

- hw drivers where the selective upload happens after compositing, i.e.
  edp psr and dsi manual upload. Generally you get 1 rectangle, maybe 2 if
  they're well-seperated enough. Screen coordinates only, and for damage
  calculation we must take all compositing parameters into account.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch