SCM_RIGHTS: XTrans vs ssh forwarding with MIT-SHM clients

[Jeremy Huddleston Sequoia]

A user reported (https://github.com/XQuartz/XQuartz/issues/314) that running `feh` on a remote system would hang the process without producing any windows when they updated from an older XQuartz server to a newer one.

I traced this to a difference in autoconf vs meson builds.  With meson, we're setting XTRANS_SEND_FDS whereas with autoconf, we weren't:

if cc.has_header_symbol('sys/socket.h', 'SCM_RIGHTS')
  conf_data.set('XTRANS_SEND_FDS', '1')
endif

vs 

        linux*|solaris*|freebsd*|dragonfly*|openbsd*)
                XTRANS_SEND_FDS=yes
                ;;
        *)
                XTRANS_SEND_FDS=no
                ;;

This change certainly looks fine to me.  darwin supports SCM_RIGHTS.  It was probably just overlooked in that configure.ac condition, and it never caused enough of a problem for someone to notice.

So I turned my attention to figuring out why things aren't working with XTRANS_SEND_FDS set...

Soon after launching `feh`, it enters ProcShmCreateSegment.  With local and unix connections, we try using SCM_RIGHTS to send an fd in the reply (_XSERVTransSocketSendFd).  With inet, inet6, and tcp, we skip sending the FD (_XSERVTransSocketSendFdInvalid). This all sounds fine and good except that ssh forwarding throws a wrench into things...

When we use ssh forwarding with a local DISPLAY (eg: `DISPLAY=:0 ssh -Y some.host`), the X11 server ends up seeing this as a local connection (thus using _XSERVTransSocketLocalFuncs) and uses SCM_RIGHTS to send a fd.  On the server, we successfully send the message with FDs attached.  On the client, we receive a reply, but there is no fd, so xcb continues to wait (specifically, read_fds() fails and we end up stuck in wait_for_reply()).

Now, I haven't dug into what's happening between the server and client, but I suspect OpenSSH just drops the FDs on the floor without logging a warning to the user and passes the rest of the message along.

So there seem to be two issues here:

1 - libxcb should recover from this.
2 - The server should be able to determine that the transport does not support SCM_RIGHTS.

---

How should this work?  Why hasn't this been reported as an issue on other platforms?  This all seems pretty platform agnostic, so I'd expect this to be an issue on other platforms as well.  Is it not?  If not, why not?

Thanks,
Jeremy