Best practices for client side buffer management

[Brad Robinson]

Hi All,

@Guillermo: yep, that's exactly the same problem I'm thinking about.

I had an idea that I'm wondering about....  the rendering model in my
toolkit is essentially the same as Windows/OSX - where the application
invalidates part of the window, and then at some later point the OS calls
back with a message prompting the app to paint that part of the screen.
ie: multiple invalidate calls are coalesced into a single paint call at a
later time - typically just before entering the message loop's idle state.

Could a Wayland app mimic this behaviour by having a single background
buffer that contains the current window content?  When the application
invalidates the window it does one of two things depending on whether the
buffer has been returned from the compositor yet.

* If the buffer has been returned and no longer in use by the compositor,
it queues a paint message to itself.

* If the buffer hasn't been returned and is still in use by the compositor,
set a flag on the window.  When the buffer is returned from the compositor,
it checks that flag and if set then it clears the flag and queues the paint
message.

The paint message redraws all the invalidated (damaged) rectangles and
re-submits it to the compositor and the whole cycle starts again.

Obviously there's more logic around maintaining the dirty region,
coalescing multiple invalidate/damage calls, an assumption that there's a
mechanism to post a message via the message loop etc... but I hope this
explains the idea.

Would this work?  I think the main requirement would be that:

1. the compositor doesn't change the contents of the buffer and that when
it's returned it's still got the old content.
2. that the compositor returns buffers it's finished with in a timely manner

It's not clear to me from what I've read that either of these points are
true or safe assumptions.

This would eliminate the need to manage multiple buffers, multiple dirty
regions, the need to copy previously rendered content between buffers and
fits nicely with the rendering model of Windows/OSX giving similar
semantics on all platforms.

Brad

On Wed, Jun 24, 2020 at 6:11 PM Guillermo Rodriguez Garcia <
guille.rodriguez at gmail.com> wrote:

> Hi Brad,
>
> El vie., 19 jun. 2020 a las 5:24, Brad Robinson
> (<brobinson at toptensoftware.com>) escribió:
> [...]
> >
> > Finally, the toolkit already maintains an off-screen buffer with the
> window's current contents rendered into it.  I'll probably replace that
> with a Wayland buffer, but wondering about partial updates.  eg: if the
> client only needs to redraw a part of the window what's the correct process
> to update just that part with Wayland.  Can I just update the existing
> buffer and prompt Wayland to just redraw that part?
>
> I am facing a similar situation with a toolkit that maintains an
> off-screen buffer with the current window contents, and will update
> that buffer at arbitrary times, when the application needs to update
> the UI.
>
> You must consider the following:
>
> Once you submit a buffer to Wayland you cannot touch it anymore until
> the compositor releases it back (wl_buffer.release callback). From
> Pekka's email:
>
> > When you submit a buffer to a Wayland compositor, it gives the
> > compositor permission to read the buffer at any time, as many times as
> > it wants to, until it tells you with wl_buffer.release that it will not
> > be looking at that buffer again. You must not write to a buffer that
> > might be read by the compositor, as that can cause misrendering on
> > screen (e.g. your repaint is shown unfinished).
>
> This means you cannot simply use the same off-screen buffer that the
> toolkit maintains, since the toolkit should not update the buffer once
> it submits it to Wayland.
>
> At some point I thought that I could have separate Wayland buffers and
> copy damaged regions from the toolkit buffer to one of these Wayland
> buffers before submitting it to the compositor.
> The idea would be to copy the set of regions that were damaged since
> the last update.
>
> But just copying damaged regions is not enough either. From Pekka's email:
>
> > Also, every buffer you submit must be fully drawn, also outside of the
> > areas you mark as damage. The compositor may be reading more than just
> > the damage you submit.
>
> So this means that everytime the toolkit updates the off-screen
> buffer, I may need to copy _the entire thing_ to a Wayland buffer and
> submit it to the compositor... not very efficient I guess.
>
> I have the feeling that Wayland is designed for a model where the
> client renders the entire frame on demand (most of the examples I've
> found so far work like this) and not for a model where the client is
> updating a buffer asynchronously and you just want to tell the
> compositor to update parts of the frame.
>
> I would be very interested to learn about your solution to this problem.
>
> Best,
>
> Guillermo Rodriguez Garcia
> guille.rodriguez at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20200624/93cbc73b/attachment-0001.htm>