why not flow control in wl_connection_flush?

[Pekka Paalanen]

On Thu, 29 Feb 2024 16:32:26 -0500
jleivent <jleivent at comcast.net> wrote:

> On Tue, 27 Feb 2024 11:01:18 +0200
> Pekka Paalanen <pekka.paalanen at haloniitty.fi> wrote:
> 
> 
> > > But suppose I'm writing a client that has the possibility of
> > > sending a rapid stream of msgs to the compositor that might be,
> > > depending on what other clients are doing, too fast for the
> > > compositor to handle, and I'd like to implement some flow control.
> > > I don't want the connection to the compositor to sever or for the
> > > condition to cause memory consumption without any ability for me to
> > > find out about and control the situation.  Especially if I could
> > > slow down that rapid stream of updates without too high a cost to
> > > what my client is trying to accomplish.
> > > 
> > > Is there a way I could do that?    
> > 
> > Get the Wayland fd with wl_display_get_fd() and poll it for writable.
> > If it's not writable, you're sending too fast.  
> 
> That's what I was looking for!  I think... Maybe?
> 
> > 
> > That's what any client should always do. Usually it would be prompted
> > by wl_display_flush() erroring out with EAGAIN as your cue to start
> > polling for writable. It's even documented.  
> 
> But, calling wl_display_flush too often is bad for throughput, right?

I don't think it is if you're not calling it in a busy-loop? Have a
look through its implementation.

If you mean calling after every single request sending, then yeah, I
would not do that, even though I don't think it would have measurable
performance impact. Power consumption impact might be measurable.

There is a wl_display mutex though, so if you hammer it from many
threads, I guess it could hurt.

> Isn't it better to allow the ring buffer to determine itself when to
> flush based on being full, and batch send many msgs?  Obviously
> sometimes the client has nothing more to send (for a while), so
> wl_display_flush then makes sense.  But in this case, it does have more
> to send and wants to know if it should attempt to do so or hold back.

Normally, your program returns to its main event loop relatively fast.
Definitely between each drawn frame if we consider just one wl_surface
at a time. That is the natural place and granularity to flush. You
must flush before polling the Wayland fd for readable, because
otherwise you might be waiting forever for replies that never come
because the compositor never got your request yet.

The automatic flush on the libwayland send buffer (which is 4 kB so
far) getting full is just to buy you time to do your work and return to
your main event loop to flush and sleep. This flush is the problematic
one, because if it fails, there is no recovery at this time. We cannot
return a "hold up, do something else for a while instead of sending
requests", and there is no place to put the current request, so the
only way out is to disconnect.

The socket buffer usually defaulting to something like 200 kB is a
saviour, I believe.

> I could instead wait for the display fd to be writable before
> attempting each msg send.  But the display fd may be writable merely
> because the ring buffer hasn't tried flushing yet.  But the ring buffer
> could have less than enough space for the msg I'm about to send.  And
> the socket buffer could have very little space - just enough for it to
> say its writable.
> 
> Which means that sometimes polling the display fd will return writable
> when an attempt to send a msg is still going to result in ring buffer
> growth or client disconnect.

Right. So you need to try to flush first to ensure the libwayland ring
buffer is empty and can hold your message. You cannot send messages
larger than the ring buffer anyway.

> So... back to calling wl_display_flush ... sometimes.
> 
> I guess I could call wl_display_flush after what I think is about 4K
> worth of msg content.  Wl_display_flush returns the amount sent, so
> that helps keep the extra state I need to maintain.
> 
> Is there currently a way I could get the size of contents in the output
> ring buffer?

I don't recall any.

The MRs for solving this problem have considered adding API for the app
to see when the ring buffer is getting a little too full.

The real problem here is though, how do you architect your app or
toolkit so that it can stop and postpone what it is doing with Wayland
when you detect that the socket is not draining fast enough? You might
be calling into Wayland using libraries that do not support this.

Returning to the main event loop is the natural place to check and
postpone, but this whole issue stems from the reality that apps do not
return to their main loop often enough or do not know how to wait with
sending even in the main loop.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20240301/cacaeb5a/attachment.sig>