[RFC wayland-protocols] inputfd - direct input access protocol

Wed Apr 5 09:35:22 UTC 2017

On Wed, 2017-04-05 at 12:40 +0900, Carsten Haitzler wrote:
> On Tue, 04 Apr 2017 14:45:13 +0200 Bastien Nocera <hadess at hadess.net>
> said:
> 
> > On Tue, 2017-04-04 at 20:32 +0900, Carsten Haitzler wrote:
> > > On Tue, 4 Apr 2017 10:22:24 +1000 Peter Hutterer <peter.hutterer@
> > > who-
> > > t.net>
> > > said:
> > > 
> > > > On Tue, Apr 04, 2017 at 08:16:03AM +0900, Carsten Haitzler
> > > > wrote:
> > > > > On Fri, 31 Mar 2017 17:29:19 +1000 Peter Hutterer
> > > > > <peter.hutterer
> > > > > @who-t.net>
> > > > > said:
> > > > > 
> > > > > If you're going to do this ... why not just do it for
> > > > > keyboards,
> > > > > mice, touch
> > > > > panels etc. too?
> > > > 
> > > > all these have desktop interactions that the compositor needs.
> > > > joysticks
> > > > generally don't, the compositor doesn't care about the events.
> > > 
> > > if the gamepad is the only input you have... then the exact same
> > > things would
> > > apply. home button, power button, menu button on a
> > > gamepad/joystick
> > > etc. would
> > > have the same use cases as alt+tab , f1, f2, ctl+alt+del etc.
> > > etc.
> > 
> > There's two possible solutions to this problem:
> > - evdev gives you the ability the mask certain events. The
> > compositor
> > can keep one fd open masking everything but the power/menu/etc.
> > buttons, and pass an fd to the app with just the power/menu/etc.
> > buttons masked. The compositor can then choose to do something
> > special
> > with those buttons
> > - you can specify a new fd_type, as mentioned in the spec, which
> > your
> > application would need to know how to handle. That can be used to
> > implement the simpler protocol that got sent a couple of months
> > ago.
> 
> then explain to me the argument where for example keyboard input
> should NOT
> also then do the same thing?

Possibly because there are more applications and toolkits that would
need keyboard input implementations than the ones requiring joypad
inputs (you'd cover 90% of the gamepads use cases with a patch to SDL),
possibly because it's grandfathered in.

I don't see any reason to keep on bringing this strawman, it might be
useful data, but it's not a good enough reason to go backwards, and
drop the load onto application and toolkit developers.

> > > > > My point is - the same reasons these other devices don't be
> > > > > lazy
> > > > > and stick
> > > > > their heads in the sand with these other devices are the same
> > > > > reasons
> > > > > joysticks, gamepads etc. etc. should also be UNDERSTOOD by
> > > > > the
> > > > > compositor
> > > > > before passing them on so they compositor can filter out
> > > > > special
> > > > > buttons
> > > > > (eg a power button and handle it internally, or a "home"
> > > > > button
> > > > > to go back
> > > > > to the home screen etc.).
> > > > > 
> > > > > The "I'll just pass the fd and otherwise have no idea what is
> > > > > going on" is
> > > > > sticking your head in the sand and avoiding a solution.
> > > > 
> > > > hence the proposal for a libgamedev to be able to share some of
> > > > that code
> > > > where needed, or to get some less device-specific handling on
> > > > the
> > > > client
> > > > side.
> > > 
> > > then how do you filter buttons and remove them from the stream
> > > other
> > > than
> > > creating a FAKE input device (eg a socketpair you pass one end to
> > > a
> > > client)...
> > > then ioctls() that would otherwise work on a joystick won't work
> > > here. if we
> > > ignore the ioctl's then faking a device and the device
> > > protocol.... 
> > 
> > See above.
> > 
> > > if that's
> > > good enough then why don't we do this for kbd and mouse too? pass
> > > fd's and fake
> > > a kbd or mouse device from the compositor if we want to filter...
> > > (yes yes i
> > > know about mouse coordinate remapping...).
> > 
> > A couple of reasons. The devices are probably "simpler", or at
> > least
> > protocol developers (be it for X11 or Wayland) felt they were.
> 
> They are not really much different.
> 
> > The problem space is better understood, and while the kernel takes
> > care
> > of a lot of the differences between devices (at least we don't need
> > to
> > figure out which keycode corresponds to which US layout keysym),
> > there
> > were still tons of problems that we ran into as those devices
> > changed:
> > - we couldn't have more than 256 different keys (minus a bunch of
> > reserved ones) in X, breaking a lot of multimedia keys.
> 
> umm actually it was 128. the MSB was press vs release. and having
> more keys
> this was solved via escapes (send multiple codes in a row with a
> special
> beginning escape value) from memory.

No, I was actually thinking about the maximum evdev keycode, which
required things like:
https://bugs.freedesktop.org/show_bug.cgi?id=31300
https://bugs.freedesktop.org/show_bug.cgi?id=31333
which required changes in the kernel, udev, and handlers at the session
level.

Fortunately, this is fixed by Wayland.

> > - similarly, the pointer protocol restricted mice to a certain
> > number
> > of buttons (10 iirc) that gaming mice broke through.
> > - a lot of functionality wasn't possible, such as per-input
> > keyboard
> > keymaps, different acceleration settings for different pointer
> > devices.
> > 
> > I'm pretty sure that we'd handle tablets similarly to the joypad
> > proposal if it weren't for the fact that the kernel support is
> > quite
> > young, and we didn't need to carry a lot of metadata about the
> > devices
> > in userspace (see libwacom).
> > 
> > > gamepads/joysticks are only "special" if your view of the world
> > > is
> > > one of
> > > "everything is a pc with a keyboard and mouse and joysticks etc.
> > > are
> > > just
> > > occasionally used input devices only for some specialist games".
> > > 
> > > that view of the world is really limited. smart tv's, consoles,
> > > handheld game
> > > systems and many more do not share this view of the world. in
> > > these
> > > worlds
> > > gamepads etc. are often primary input devices that do act as
> > > surrogate
> > > keyboards and mice along with other stuff (conceptually if your
> > > view
> > > of the
> > > world is that of a pc).
> > 
> > It really isn't, sorry. In fact, I'm pretty certain that you can
> > implement a gaming console's interface far more easily with this
> > protocol than with the original one.
> 
> how so? all a gamepad/jostick etc. protocol has to do is pass on the
> same data
> as the kernel device.

You'd lose the filtering features through ioctl(), and the ability to
gather metadata about the device, as mentioned in the part of the
thread discussing properties. I don't see the point of having a pair of
file descriptors instead of just one for this case.

>  unlike a kernel device it'd at least be portable as a
> different OS may or may not support the same devices. if we're doing
> the "well
> we'll emulate a linux evdev device then on all platforms" then why
> not just do
> so with keyboard input too? why not so this everywhere and just have
> an fd per
> input device rather than wayland protocol.

Non-Linux OSes would likely add an fd_type, and add support for that
fd_type to their toolkits.

> it's all inconsistent this way. some input is protocol. some is a
> special
> fd with a device-driver level protocol not a wayland one.
> 
> if the kernel devices are so well done to "just expose them" as a fd
> then why
> not simply transfer that protocol into a wayland protocol event and
> be done
> with it? at least we're then consistent with the way input events are
> done.
> that or dump wayland protocol input device events and do them the
> device fd
> way...

Because we'd still need to define that protocol, and extend it every
time the evdev kernel protocol itself is extended. I'd rather rely on
the kernel to provide that API, meaning we wouldn't need to constantly
update the protocol, and any refinement, or clarification is the
kernel's.

If you're willing to filter the kernel's stream, and marshall the
events in some other way, please feel free to propose a new fd_type and
document your protocol. My 3 different use cases for this protocol
won't need that layer of translation.

Cheers