[Xcb] Help with external socket handling and xcb

[Uli Schlachter]

Hi,

my mail program just informed me that there are new mails in my "xcb"
folder. No idea why it needed ten days to do so. Sorry!

Uhm. There is a lot going on in your mail. I'll skip all the "I wish
things where like XYZ", because I don't really know what to answer to
that. Sorry, but things are not like that currently.

Am 09.09.21 um 12:48 schrieb Carlo Wood:
[...]
>>> So, since it doesn't do that (as far as I know), it will have to
>>> make sure that every time it has something to write, it writes
>>> ALL of it before returning to my code... and that sounds very
>>> blocking. How can this ever work reliable with a non-blocking
>>> fd? Or is xcb just calling write in a loop until it wrote everything
>>> it has to write (also when write returns EAGAIN)?  
>>
>> Basically yes. _xcb_conn_wait() calls write_vec() to write some data
>> whenever poll()/select() say that the FD is writeable and there is
>> some pending data to send. Here is the loop that then calls
>> _xcb_conn_wait() until everything was written:
>> https://github.com/freedesktop/xcb-libxcb/blob/ee9dfc9a7658e7fe75d27483bb5ed1ba4d1e2c86/src/xcb_out.c#L457-L458
> 
> Just to be clear; so this means that in order for xcb_poll_for_event
> to work while only calling it when I see that the fd is readable, I only
> ever have to monitor that the fd is readable, and never have to worry
> about monitoring if the fd is writable, because all writing is done
> blocking, even though the fd itself is non-blocking?

Yup. You just monitor the FD for readability and then you can use
xcb_poll_for_event() / xcb_poll_for_reply() to check for "what is going
on". There is also xcb_poll_for_event_queued() which only checks the
internal buffer, but does not try to read anything from the socket.

But beware: Any XCB function can read from the socket. Even sending a
request may read from the socket (this is necessary to avoid a deadlock
with the X11 server). Thus, you should "pretend" that the socket is
readable after you wrote anything and call xcb_poll_for_event() /
xcb_poll_for_reply() afterwards.

> 
>>
>>> If anyone can give me pointers, or maybe tell me about a part of
>>> the API that I missed that is suitable for this, that would be
>>> great!  
>>
>> For non-blockingly waiting for a reply to a request: Do you know about
>> xcb_poll_for_reply()? There is an example on how to use that to get a
>> callback-based "your reply arrived"-API here:
>> https://gitlab.freedesktop.org/xorg/lib/libxcb/-/issues/52#note_712068
>> (Hm, it seems like this code does not handle sequence number
>> overflows...)
> 
> Hmm. This is still polling in nature and hard to wrap my head around
> without knowing more about the protocol that is being sent to and from
> the X server.

If you want to know more about the protocol and want to know how to
identify individual packages:

- Stuff being sent to the server has a length field in bytes 3/4 of the
packet. This is the number of four-byte-units of the packet.
- There is an extension for "big requests". In this case, the package
length will be zero and the following four bytes form a 32 bit length field.

- Stuff being received from the server always has at least 32 bytes.
- The first byte of a packet is its response_type
- Errors have response_type 0 and are always 32 bytes large
- Replies have response_type 1 and xcb_generic_reply_t describes their
layout. The length field is the
number-of-four-byte-quantities-after-the-32-bytes. So, a reply has
length 32 + 4*((xcb_generic_reply_t*)r)->length
- Events always have 32 bytes
- Except that this turned out not to be enough and the X11 generic
events extension was born. These events have response_type=35 and again
contain a length field (see xcb_ge_generic_event_t)

See here:
https://github.com/freedesktop/xcb-libxcb/blob/ee9dfc9a7658e7fe75d27483bb5ed1ba4d1e2c86/src/xcb_in.c#L236-L254

> Really non-polling would be when a callback for arrived replies is done
> whenever the function that reads incoming data (called by my
> application when there is data to read on the fd) sees that such
> replies did arrive.

Okay, then when the socket is readable, you do something like
dispatch_replies(conn, 1) from the code here:
https://gitlab.freedesktop.org/xorg/lib/libxcb/-/issues/52#note_712068
Additionally, you use xcb_poll_for_event() to check if an event was
received.

> I could call "poll_for_reply" whenever the socket becomes readable, but
> that makes no sense because at that moment the socket hasn't been read
> yet.

Both xcb_poll_for_event() and xcb_poll_for_reply() will try to read from
the socket when:

- there is no pending event/reply in the buffer and
- no other thread is currently trying to read from the socket

xcb_poll_for_queued_event() skips the "try to read from the
socket"-part. It was added because this unconditional trying to read
from the socket caused slowdowns with libX11. There is no
xcb_poll_for_queued_reply() currently.

Cheers & again sorry for the slow answer,
Uli
-- 
- He wants the impossible...!
- That's the short definition of 'captain'.