[Xcb] Remaining Xlib/Xcb problems

[Rami Ylimäki]

  Hi,

I'd like to try solving 
https://bugs.freedesktop.org/show_bug.cgi?id=29561 properly and 
eliminate excessive non-blocking reads done by Xlib. Jamey hinted in the 
bug comments that the best way to approach this problem would be to 
implement a response queue in Xcb and modify Xlib to use that instead of 
current Xcb interface.

I've been spending some time reading Xcb and Xlib code in order to 
clarify myself how to fix the problem. It has become clear that the 
public interface of Xcb needs to be extended and an alternative for 
xcb_poll_for_event is needed because that function may do a non-blocking 
read when it's not necessary. My understanding from Jamey's comment is 
that any interface changes are not likely to be accepted unless the 
changed interface can also be used to facilitate solving of remaining 
Xlib problems at least in principle.

In order to get the original problem fixed, I need to first understand 
what the remaining Xlib problems are and how the response queue solution 
suggested by Jamey can be used to solve them. Before thinking about the 
details of a response queue implementation, I'd like to get a 
confirmation from you about the known problems. Below is a list of 
relevant problems that I could find based on Bugzilla comments, commit 
messages and comments embedded in code.

   1. Concurrent event waiters can prevent reply waiters from advancing 
until an event arrives. This is caused by waiting on 
_XReply:dpy->xcb->event_notify even in cases where the reply waiter 
could advance.

   2. Requests that expect multiple replies without an asynchronous 
reply handler aren't thread safe. Examples of this kind of request are 
XListFontsWithInfo and XRecordEnableContext. This is caused because 
_XReply calls assume that a reply to the latest request is expected. 
However, other threads can add more pending replies while the 
problematic request is sleeping unlocked in _XReply:xcb_wait_for_reply. 
After waking up, the problematic request exits _XReply but calls it 
again in a loop until a last reply has been received. If more pending 
requests have been queued in the meanwhile, it's possible to lose track 
of the correct pending reply. Because of this, replies will be 
incorrectly skipped and reply waiters may possibly start to read replies 
to wrong requests.

   3. Sometimes a non-blocking read system call is done even though it 
wouldn't be necessary. This is caused by _XReply:poll_for_event calls, 
which check the connection for new data even though this is not 
necessary in called from _XReply.

Is there something else that I should be aware of? Did I misunderstand 
some of the problems listed above (I didn't actually write test cases to 
verify them)?

Once there is an agreement of the remaining problems, we can start to 
design a response queue that would best address the issues. I guess it 
could be as simple as providing a way to extract responses based on 
sequence numbers from Xcb buffers to a separate queue and allow 
peeking/iteration the contents of the queue without removing elements. I 
guess the main problem with the current Xcb interface from Xlib point of 
view is that it's not possible to examine what has been received over 
the connection without simultaneously extracting responses from the Xcb 
buffers. Has anyone already thought about implementing this and 
identified some special requirements for the response queue?

-- Rami