Best practices for client side buffer management

Sebastian Wick sebastian at
Wed Jun 24 11:35:12 UTC 2020

On 2020-06-24 13:14, Pekka Paalanen wrote:
> On Wed, 24 Jun 2020 19:17:57 +1000
> Brad Robinson <brobinson at> wrote:
>> Hi All,
>> @Guillermo: yep, that's exactly the same problem I'm thinking about.
> Hi,
> I answered to Guillermo as well further down.
>> I had an idea that I'm wondering about....  the rendering model in my
>> toolkit is essentially the same as Windows/OSX - where the application
>> invalidates part of the window, and then at some later point the OS 
>> calls
>> back with a message prompting the app to paint that part of the 
>> screen.
>> ie: multiple invalidate calls are coalesced into a single paint call 
>> at a
>> later time - typically just before entering the message loop's idle 
>> state.
>> Could a Wayland app mimic this behaviour by having a single background
>> buffer that contains the current window content?  When the application
>> invalidates the window it does one of two things depending on whether 
>> the
>> buffer has been returned from the compositor yet.
>> * If the buffer has been returned and no longer in use by the 
>> compositor,
>> it queues a paint message to itself.
>> * If the buffer hasn't been returned and is still in use by the 
>> compositor,
>> set a flag on the window.  When the buffer is returned from the 
>> compositor,
>> it checks that flag and if set then it clears the flag and queues the 
>> paint
>> message.
> The problem is, the compositor might not release the buffer until
> you have already submitted a new one.
> A compositor, that textures directly from the buffer you submitted as a
> client, cannot release the buffer until it gets a new buffer to replace
> it. A compositor may repaint at any time for any reason, so it must
> always have window content (some buffer) available. Compare this to X11
> expose events: expose events require the client to draw the damaged
> area and until it does, the display will have garbage there. A Wayland
> compositor handles expose events completely internally, which means
> that no such garbage is ever shown on display.
> You can of course keep a shadow buffer as the main buffer, and copy
> appropriately to Wayland buffers when you update, but it consumes an
> extra buffer obviously.
> It would be preferable if your widget tree / scenegraph was able to
> draw arbitrary regions from scratch to arbitrary buffers on-demand, and 
> not
> autonomously at "random" times, so you can draw directly into Wayland
> buffers exactly the parts that need drawing. Then you would also never
> throw any drawing away without showing it, so it's also an energy usage
> optimization.
> It's not actually that different from the window invalidation you
> describe, except it will be all completely internal to your toolkit and
> "the window" is actually "a buffer", and you need to keep track of
> damage history as well. More on that below.
>> The paint message redraws all the invalidated (damaged) rectangles and
>> re-submits it to the compositor and the whole cycle starts again.
>> Obviously there's more logic around maintaining the dirty region,
>> coalescing multiple invalidate/damage calls, an assumption that 
>> there's a
>> mechanism to post a message via the message loop etc... but I hope 
>> this
>> explains the idea.
>> Would this work?  I think the main requirement would be that:
>> 1. the compositor doesn't change the contents of the buffer and that 
>> when
>> it's returned it's still got the old content.
> With Wayland, wl_surface.attach does not allow the compositor to write
> into the buffer, it only allows reading. So that is safe, also because
> the client manages its own buffers (allocate, re-use, destroy).

Isn't this a little bit more subtle? In particular if you use OpenGL in
the compositor to access the image there might be layout transitions
which change the image in place.

So while the buffer is owned by the compositor the client must not read
or write to it and when the ownership is transferred back to the client
the image might be in another layout.

Or am I missing something here?

>> 2. that the compositor returns buffers it's finished with in a timely 
>> manner
> This is false as described above.
>> It's not clear to me from what I've read that either of these points 
>> are
>> true or safe assumptions.
>> This would eliminate the need to manage multiple buffers, multiple 
>> dirty
>> regions, the need to copy previously rendered content between buffers 
>> and
>> fits nicely with the rendering model of Windows/OSX giving similar
>> semantics on all platforms.
> Unfortunately you cannot avoid multi-buffering in a client that aims to
> be correct.
> If Wayland dictated that clients must be able to run fine with just one
> buffer per window, then quite likely the compositor would be forced to
> maintain a full copy of the window contents, or it would not be able to
> refresh the screen at all times, or it might end up showing
> transitional garbage (glitch).
> The Wayland design principle is "every frame is perfect", which more or
> less leads to "never show garbage", and "never stall waiting for
> clients to respond".
>> On Wed, Jun 24, 2020 at 6:11 PM Guillermo Rodriguez Garcia <
>> guille.rodriguez at> wrote:
>> > Hi Brad,
>> >
>> > El vie., 19 jun. 2020 a las 5:24, Brad Robinson
>> > (<brobinson at>) escribió:
>> > [...]
>> > >
>> > > Finally, the toolkit already maintains an off-screen buffer with the
>> > window's current contents rendered into it.  I'll probably replace that
>> > with a Wayland buffer, but wondering about partial updates.  eg: if the
>> > client only needs to redraw a part of the window what's the correct process
>> > to update just that part with Wayland.  Can I just update the existing
>> > buffer and prompt Wayland to just redraw that part?
>> >
>> > I am facing a similar situation with a toolkit that maintains an
>> > off-screen buffer with the current window contents, and will update
>> > that buffer at arbitrary times, when the application needs to update
>> > the UI.
>> >
>> > You must consider the following:
>> >
>> > Once you submit a buffer to Wayland you cannot touch it anymore until
>> > the compositor releases it back (wl_buffer.release callback). From
>> > Pekka's email:
>> >
>> > > When you submit a buffer to a Wayland compositor, it gives the
>> > > compositor permission to read the buffer at any time, as many times as
>> > > it wants to, until it tells you with wl_buffer.release that it will not
>> > > be looking at that buffer again. You must not write to a buffer that
>> > > might be read by the compositor, as that can cause misrendering on
>> > > screen (e.g. your repaint is shown unfinished).
>> >
>> > This means you cannot simply use the same off-screen buffer that the
>> > toolkit maintains, since the toolkit should not update the buffer once
>> > it submits it to Wayland.
>> >
>> > At some point I thought that I could have separate Wayland buffers and
>> > copy damaged regions from the toolkit buffer to one of these Wayland
>> > buffers before submitting it to the compositor.
>> > The idea would be to copy the set of regions that were damaged since
>> > the last update.
>> >
>> > But just copying damaged regions is not enough either. From Pekka's email:
>> >
>> > > Also, every buffer you submit must be fully drawn, also outside of the
>> > > areas you mark as damage. The compositor may be reading more than just
>> > > the damage you submit.
>> >
>> > So this means that everytime the toolkit updates the off-screen
>> > buffer, I may need to copy _the entire thing_ to a Wayland buffer and
>> > submit it to the compositor... not very efficient I guess.
> You don't have to copy everything.
> There is an EGL extension called EXT_buffer_age, which defines the
> concept of "buffer age". The concept is important, not that it's EGL.
> The concept is very applicable also without EGL.
> Roughly said, it is how many updates ago that particular buffer was
> last current (sent to the compositor). (Check the detailed
> specification for accurate description, my hand-waving may be
> imprecise.) The main idea is that you keep a history of damage, and
> when you re-use a buffer that already has content from 3 frames ago,
> you combine the damage history over the three frames to know what parts
> you need to paint. That way you don't need to paint/copy everything
> every time.
> Mind, that what you paint (buffer damage) is not what you submit as
> damage via Wayland (surface damage). Buffer damage is what you have to
> paint in the buffer to make it completely up-to-date. Surface damage is
> what changes on the surface in this update compared to its current
> contents on the compositor.
>> >
>> > I have the feeling that Wayland is designed for a model where the
>> > client renders the entire frame on demand (most of the examples I've
>> > found so far work like this) and not for a model where the client is
>> > updating a buffer asynchronously and you just want to tell the
>> > compositor to update parts of the frame.
> Wayland does support partial damage as explained above. It just needs a
> little more damage tracking that you might be used to.
> contain some useful explanation on surface and buffer damage. Again,
> the concepts apply also outside of EGL.
> OTOH, it is very true that on Wayland, the compositor should be the one
> triggering your draws *if* you already have something new to show. A
> client can draw any time it wants, and it can post a new frame any time
> it wants, that's all ok. But, if a client draws a frame that will never
> be displayed (because the client draws another frame too soon so the
> compositor just discards the first one), then it will waste precious
> energy.
> wl_surface.frame callback exists so that no client would do useless
> work by drawing frames that will be discarded.
> Thanks,
> pq
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at

More information about the wayland-devel mailing list