[RFC wayland-protocols] inputfd - direct input access protocol

[Peter Hutterer]

On Fri, Mar 31, 2017 at 12:23:49PM +0300, Pekka Paalanen wrote:
> > +      <description summary="device capability notification">
> > +	This event is sent to notify the client of a custom property that
> > +	applies to this device. The property is a standard key/value store
> > +	in UTF-8 format, interpretation of both strings is left to the
> > +	client. The wayland protocol makes no guarantees about the content
> > +	of each string beyond its text encoding.
> > +
> > +	Compositors and clients need to agree on a dictionary of properties.
> > +	For example, a compositor may designate the device to be of
> > +	'joystick-type' 'gamepad'. This dictionary is out of the scope of
> > +	this protocol.
> > +      </description>
> > +      <arg name="property" type="string" summary="A UTF-8 encoded property name"/>
> > +      <arg name="value" type="string" summary="A UTF-8 encoded property value"/>
> > +    </event>
> 
> While I understand the desire to leave the dictionary for others to
> specify, this event is essentially useless without it. The dictionary
> really is part of the protocol, even if you didn't want it. I'm not
> sure it's helpful to leave the authority on the dictionary unspecified.
> As you wrote, they need to be agreed somewhere.
> 
> This raises a couple of questions for the protocol itself:
> 
> - What should the client do with a property or value it does not
>   recognize?
> 
> - If the client does not recognize a property or a value, can it still
>   use the device? Could this vary per property?
> 
> - Should the compositor be sending only properties and values the
>   client can understand?
> 
> If this protocol does not define a single authority on the dictionary,
> should it then carry a dictionary type, id or namespace?
> 
> Should properties be namespaced?

ok, easy answers first :)

clients must ignore property names or values that they do not understand,
the compositor may send any value (it won't know what the client supports),
the protocol does not enforce namespacing, but the dictionary authority may
do so.
one more thing I thought of: properties key/values are *not* singular, a
client may receive the same property with multiple different values (e.g.
ORIENTATION=left, ORIENTATION=top or BUTTON=A, BUTTON=X, ...). Whether
that's allowed for any property depends on the dictionary.

Now to the difficult answers:

I think it'll be impossible to *not* have this in an external dictionary.
The needs of the compositor and the clients are both the same and completely
different for this protocol to work correctly. Let's take the example of
tagging a device as gamepad. The default udev rules are unreliable,
ID_INPUT_JOYSTICK has too many false positives. So we need some database
that tags the device as JOYSTICK_TYPE=gamepad, JOYSTICK_TYPE=wheel, etc.
That can be done in udev, but needs some external authority anyway -
libwacom style. Without that, you'll have a mismatch in what compositors
detect as gaming devices vs what clients can handle (assuming the mythical
libgamingdevice exists).

The compositor doesn't care about the actual value though, anything with
JOYSTICK_TYPE is a gaming device and will be accessible through inputfd.
Adding new joystick types in the future does not change the protocol or the
compositor implementation. A client should not need to wait for protocol bumps just
to get access to a new type that the compositor doesn't care about anyway.
Just as a general reminder, the whole point of this protocol is for the
compositor to have some say in what's available but otherwise get out of the
way.

If we don't have the generic properties, we need the client to have some
other way to access information about the device. In the Wacom case we have
'path' which gives the client a syspath and thus the udev device and access
to udev properties. Bastien wasn't too happy about this for GPIO devices
where it may not be possible, so the property approach was the first thing
that came to mind.

Having said this - while I don't want this in the protocol, there's no
reason we can't define that dictionary authority ourselves. I just want it
external to the protocol. ICCCM/EMWH-style, effectively. But I would feel
uncomfortable *being* the authority, given that this is too far outside my
core knowledge. Happy to be the maintainer for this though, an initial
implementation would be largely boilerplate.

I still think that there's a market for a generic libgamingdevice library
that works libinput-style for gaming devices and provides the database that
matches this dictionary. 


> > +    <event name="focus_in">
> > +      <description summary="input fd device focus in event">
> > +	Notification that this client now has the focus and/or access to
> > +	this device. The decision what consitutes focus left to the
> > +	compositor. For example, a compositor may tie joystick focus to the
> > +	wl_pointer focus of this seat. The protocol does not guarantee that
> > +	any specific client ever receives the focus for a device.
> > +
> > +	The client is passed a file descriptor with access to this
> > +	device. This file descriptor is valid until a subsequent
> > +	wp_inputfd_device.focus_out event. Upon wp_inputfd_device.focus_out, the
> > +	compositor may revoke the fd and further operations will fail.
> > +
> > +	However, due to potential race conditions a client must be able to
> > +	handle errors as if it opened the fd itself. No guarantee is
> > +	given that the wp_inputfd_device.focus_out event or wp_inputfd_device.removed
> > +	event are sent before the client encounters an error on the file
> > +	descriptor.
> > +
> > +	A compositor guarantees that the underlying device does not change
> > +	until a wp_inputfd_device.removed event. In other words, if the fd
> > +	type allows querying capabilities through the fd, a client needs to
> > +	do so only once at the first focus_in. Subsequent focus_in events
> > +	will provide the same capabilities.
> > +
> > +	If applicable, this event contains the surface that has the focus.
> > +	In some cases, the focus may not be tied to a specific client surface
> > +	but is given to the client independent of any surface. In that case,
> > +	the surface is null.
> > +
> > +	The protocol guarantees that focus_in and focus_out always come in
> > +	pairs. If the client currently has the focus and the device is
> > +	removed, a focus_out event is sent to the client before the
> > +	wp_inputfd_device.removed event.
> > +      </description>
> > +      <arg name="serial" type="uint"/>
> 
> What is 'serial'?

will fix


> > +      <arg name="fd" type="fd" summary="file descriptor to the device"/>
> > +      <arg name="fd_type" type="uint" enum="fd_type" summary="fd type" />
> > +      <arg name="surface" type="object" interface="wl_surface" summary="The current surface that has the device's focus" allow-null="true"/>
> > +    </event>
> 
> This reminded me, we should probably check whether libwayland-client
> can handle 'focus_in' racing with 'destroy' without leaking the fd. I
> have no recollection if it does. That is, when the client sends
> 'destroy' at the same time the server sends 'focus_in', the client no
> longer has a live proxy (has a zombie instead), does libwayland-client
> while dropping the 'focus_in' on the floor also close the received fd.

implementation-detail? :)
but yeah, needs fixing.

> > +
> > +    <event name="focus_out">
> > +      <description summary="input fd device focus out event">
> > +	Notification that this client no longer has focus and/or access to
> > +	this device. Further reads from this device's file descriptor
> > +	will fail. The client must close(2) the file descriptor received in
> 
> "will fail" seems a bit strong.

well, in the case of EVIOCMUTE that's exactly what will happen, ENODEV
galore for anything after the ioctl. Having this in the protocol makes it
unambiguous what's going to happen and clear that any client that cannot
handle this case is buggy. "will fail soon" might be more precise, but I'm
not sure it's as clear.

Cheers,
   Peter

> 
> > +	the wp_inputfd_device.focus_in event.
> > +
> > +	This event does not mean the device was removed, merely that the
> > +	device is focused elsewhere. For device removal, see
> > +	wp_inputfd_device.removed.
> > +
> > +	See wp_inputfd_device.focus_in for more details.
> > +      </description>
> > +    </event>
> > +  </interface>
> > +</protocol>
> > +
> > +
> 
> Thanks,
> pq