<div dir="ltr"><div>Hi Peter,</div><div> </div><div>Thanks for sharing this proposal. I have some little comments for a later stage, but would rather discuss some big items first.</div><div> </div><div>The feedback I will share comes from a couple different angles as we are interested in Wayland for various different use cases.</div><div> </div><div>Originally we had the gamepad wayland thread originally started by Jingkui. The proposal there was to provide knowledge of axes, buttons and everything making up a gamepad to the compositor. He proposed this direction as a way to inject events from Chrome into Android as at that level there is no file descriptor anymore. Ourselves we were quite interested in this approach as well. Among the things we do is roughly remote streamable desktops. Currently on X you can inject keyboard/mouse events through XTest, but the same applies to a custom Wayland compositor into which you can easily inject keyboard/mouse data. Clients don't have to be aware of the custom virtual devices and protocols behind the scenes.</div><div> </div><div>When using inputfd for such a use case, either we would need to fake an input device through uinput (not ideal) or add a new 'fd_type' and have a custom protocol across this. Clients or input libraries would need to be able to deal with this other protocol, which would be a big pain. For such remote desktop use cases and even the Android one inputfd is not ideal.</div><div> </div><div>Now back to general inputfd feedback and requirements. I want to make sure basic gamepad functionality is handled and also more complex functionality. As the maintainer of hid-sony for ds3/ds4 controllers, I want to make sure our gamepads are supported well. The same feedback would also apply to xbox360, switch or steam controllers as all devices have become more complex.</div><div> </div><div>The current proposal could work for exposing basic gamepad functionality for the axes/buttons. Taking hid-sony as an example ds3 and ds4 devices are now threated as composite devices utilizing multiple device nodes. The ds4 is probably the best example using 3 evdev nodes, 1 for gamepad, 1 for touchpad (reported as a pointer device) and 1 for motion sensors. I expect the Switch Pro controller and the Joypads for the Switch to use at minimum 2 nodes as well.</div><div> </div><div>How would composite devices be handled? There would need to be a way to tie multiple devices together. So far composite devices use all the same ids (product/vendor) and same unique id (EVIOCGUNIQ) and even physical location (EVIOCGPHYS). At minimum I think the compositor needs to tie these together with some sort of shared id. Another issue again taking ds4 as an example it has a touchpad, which is already picked up as a pointer device elsewhere. How to handle this? Unflag it as a pointer and share this inputfd as well or use it really as a pointer and gamepad state the other way?</div><div> </div><div>In addition next to evdev nodes there are sysfs nodes for which fd passing won't work. The best example would be LEDs as found on ds3/ds4/xbox360, but I'm sure there will be GPIO as well. In case of ds4 it is not uncommon to update the LEDs many times a second (often used for light effects). How would such features be exposed? Is the idea to leverage properties or something like this? Passing these nodes to clients is probably not a good idea.</div><div> </div><div>This is some initial feedback. I'm not sure how much I like the inputfd proposal. For a part because it doesn't fit some of our use cases well (remote desktop + custom protocols). In addition I'm also a bit worried about the complexity in handling composite devices.</div><div> </div><div><div>Thanks,</div><div>Roderick </div>-- <div class="gmail_signature">Roderick Colenbrander Senior Manager of Software Engineering Gaikai, a Sony Interactive Entertainment Company <a href="mailto:roderick@gaikai.com" target="_blank">roderick@gaikai.com</a></div></div><div> </div><div class="gmail_extra"> <div class="gmail_quote">On Fri, Mar 31, 2017 at 12:29 AM, Peter Hutterer <<a href="mailto:peter.hutterer@who-t.net" target="_blank">peter.hutterer@who-t.net</a>> wrote: <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">This is the first draft for a protocol to enable direct access to input devices. The basic premise is: instead of protocols to handle gaming devices, 3D mice, sensors, etc. we just hand you (the client) an fd and hope you're happy with it. You get to play with that fd until it's revoked, i.e. right now we only provide it for evdev fds where we have EVIOCMUTE. Focus management is straightforward and follows compositor policy. The protocol is quite simple: clients get a wp_inputfd_device for each device this device sends some static information (name, usb ids, ...) this device sends focus-in with an fd this device sends focus-out, the fd is revoked ... repeat focus in/out as necessary ... this device sends removed on unplug The devices are wrapped up into seats and those seats are provided by a global manager object - largely because that's what we need for the protocol to work. The notable bits are: * devices are divided into categories, so instead of getting all devices, you say "give me an inputfd-seat for gaming devices". That object then sends you all devices. In the future, we'll have an equivalent "give me the seat for 3D mice", etc., but right now the gaming devices is all that exists * there's a generic key/value event called "property". This one is largely so the compositor can tag devices with some capabilities to make it easier for the client to identify what it wants. I'll punt to Bastien for details because he requested that, but I can imagine something like "joystick type" is "gamepad". This big deal with this is that the dictionary of tags is something that compositor and clients will have to agree on, and it's not something defined by the protocol. This enables us to change/extend the tags independent of protocol bumps. It's quite similar to having access to the udev device to get at udev properties, but some devices may not have a udev device, so... What is *not* in the protocol is how the compositor decides what is a gaming devices. The ID_INPUT_JOYSTICK udev property is too crude with many false positives. For tablets, we get around this by having libwacom spit out udev rules to tag things correctly. A similar approach is the best solution here: have some database that knows about various joystick devices and which category they fall into and install the udev rules/hwdb entries accordingly. There was talk of a gaming-device-oriented library (libgamingdev), this library could provide that information. Such a library would also be a prime spot for that property dictionary. Volunteers to start working on this, please step forward! /me takes a big step back Again, this is a draft, so feel free to pick it apart or request changes so it matches your use-case. Please do add CC's as required, I am by no means a game developer so I'm not aware of use-cases beyond the immediately obvious. Cheers, Peter --- diff --git a/Makefile.am b/Makefile.am index e693afa..e46910a 100644 --- a/Makefile.am +++ b/Makefile.am @@ -4,6 +4,7 @@ unstable_protocols = \ unstable/linux-dmabuf/linux-dmabuf-unstable-v1.xml \ unstable/text-input/text-input-unstable-v1.xml \ unstable/input-method/input-method-unstable-v1.xml \ + unstable/inputfd/inputfd-unstable-v1.xml \ unstable/xdg-shell/xdg-shell-unstable-v5.xml \ unstable/xdg-shell/xdg-shell-unstable-v6.xml \ unstable/relative-pointer/relative-pointer-unstable-v1.xml \ diff --git a/unstable/inputfd/README b/unstable/inputfd/README new file mode 100644 index 0000000..a24d858 --- /dev/null +++ b/unstable/inputfd/README @@ -0,0 +1,4 @@ +Input device fd passing protocol + +Maintainers: +Peter Hutterer <<a href="mailto:peter.hutterer@who-t.net">peter.hutterer@who-t.net</a>> diff --git a/unstable/inputfd/inputfd-unstable-v1.xml b/unstable/inputfd/inputfd-unstable-v1.xml new file mode 100644 index 0000000..3b83378 --- /dev/null +++ b/unstable/inputfd/inputfd-unstable-v1.xml @@ -0,0 +1,279 @@ +<?xml version="1.0" encoding="UTF-8"?> +<protocol name="inputfd_unstable_v1"> + <copyright> + Copyright 2017 © Red Hat, Inc. + + Permission is hereby granted, free of charge, to any person + obtaining a copy of this software and associated documentation files + (the "Software"), to deal in the Software without restriction, + including without limitation the rights to use, copy, modify, merge, + publish, distribute, sublicense, and/or sell copies of the Software, + and to permit persons to whom the Software is furnished to do so, + subject to the following conditions: + + The above copyright notice and this permission notice (including the + next paragraph) shall be included in all copies or substantial + portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + SOFTWARE. + </copyright> + + <description summary="Wayland protocol for direct fd access to input devices"> + This description provides a high-level overview of the interfaces + in this protocol. For details, see the protocol specification. + + Some input devices do not interact with the windowing system. Examples + of such input devices are gaming controllers or 3D mice. In many cases, + a client requires direct access to the device to access or interpret + device-specific functionality. + + This interface provides the ability for a compositor to pass a file + descriptor to the client. The compositor may restrict the type of device + designated as compatible device and it may restrict specific events from + being sent to the client (e.g. by masking the Home button on a gamepad). + Otherwise, a client should treat the device as if opened manually. + + This interface divides devices into type-based categories, clients are + expected to only request access to devices that match the category they + require. As of version 1, this interface provides only a single + category: "gaming devices". Future categories of devices are expected to + re-use the same wp_inputfd_device interface, but may differ in the way + how to request them. + + Multiple input devices of the same type may exists and assigned to + different seats. The top-level object of this protocol is a + wp_inputfd_manager. Dependent on the device type sought after, a client + may request a type-specific seat interface (e.g. + wp_inputfd_manager.get_seat_gamedev). This seat then provides the list + of devices for that category. + + Once a compositor deems a device to be focused on the client or on a + client's surface it sends a wp_inputfd_device.focus_in event with a file + descriptor for this device. A compositor may arbitrarily revoke access + to the device by sending a wp_inputfd_gamedev.focus_out. Additionally, a + compositor may invoke system functionality to restrict access to the + file descriptor, e.g. by using EVIOCMUTE on an evdev fd. + + Otherwise, a client should treat the file descriptor as direct access to + the device for the duration of it having access. + </description> + + <interface name="zwp_inputfd_manager_v1" version="1"> + <description summary="controller object for direct fd access input devices"> + An object that provides access to the input devices available for + direct fd access on this system. All input devices are associated with + a seat, to get access to the actual devices, use one of the get_seat + requests, e.g. wp_inputfd_manager.get_seat_gamedev. + </description> + + <request name="get_seat_gamedev"> + <description summary="get the seat for gaming devices"> + Get the wp_inputfd_seat object for the given seat. This object + provides access to all gaming devices in this seat. + + The decision which device is a gaming device is made by the + compositor. The protocol makes no guarantees whether a particular + device is available through this interface. + </description> + <arg name="inputfd_seat" type="new_id" interface="zwp_inputfd_seat_v1"/> + <arg name="seat" type="object" interface="wl_seat" summary="The wl_seat object to retrieve the input devices for" /> + </request> + + <request name="destroy" type="destructor"> + <description summary="release the memory for the inputfd manager object"> + Destroy the wp_inputfd_manager object. Objects created from this + object are unaffected and should be destroyed separately. + </description> + </request> + </interface> + + <interface name="zwp_inputfd_seat_v1" version="1"> + <description summary="controller object for input devices of a seat"> + An object that provides access to the input devices available on this + seat for the requested type of device. After binding to this + interface, the compositor sends a set of wp_inputfd_seat.device_added + events for currently available devices and whenever a new device + becomes available. + </description> + + <request name="destroy" type="destructor"> + <description summary="release the memory for the inputfd seat object"> + Destroy the wp_inputfd_seat object. Objects created from this + object are unaffected and should be destroyed separately. + </description> + </request> + + <event name="device_added"> + <description summary="new device notification"> + This event is sent whenever a new device becomes available on + this seat. This event only provides the object id of the devices, + any static information about the device (device name, + vid/pid, etc.) is sent through the wp_inputfd_device interface. + + Which devices are compatible input devices for this seat is a + decision made by the compositor, the protocol makes no guarantee + that any specific device becomes available as inputfd device to a + client. + </description> + <arg name="id" type="new_id" interface="zwp_inputfd_device_v1" summary="the newly added device"/> + </event> + </interface> + + <interface name="zwp_inputfd_device_v1" version="1"> + <description summary="input fd device"> + The wp_inputfd_device interface represents one device accessible + directly by an fd passed to the client. + + A device has a number of static characteristics, e.g. device + name and pid/vid. These capabilities are sent in an event sequence + immediately after the wp_inputfd_seat.device_added event. This initial + event sequence is terminated by a wp_inputfd_device.done event. This + sequence is sent only once and always before the first + wp_inputfd_device.focus_in event. + </description> + + <request name="destroy" type="destructor"> + <description summary="destroy the inputfd object"> + This destroys the client's resource for this inputfd object. + </description> + </request> + + <event name="name"> + <description summary="device name"> + The name is a UTF-8 encoded string with the device's name, intended + for presentation to the user. + + This event is sent in the initial burst of events before the + wp_inputfd_device.done event. + + This event is optional, if the required information is not available + for this device the event is omitted. + </description> + <arg name="name" type="string" summary="the device name"/> + </event> + + <event name="usb_id"> + <description summary="device USB vendor/product id"> + This event is sent in the initial burst of events before the + wp_inputfd_device.done event. + + This event is optional, if the required information is not available + for this device the event is omitted. + </description> + <arg name="vid" type="uint" summary="USB vendor id"/> + <arg name="pid" type="uint" summary="USB product id"/> + </event> + + <event name="property"> + <description summary="device capability notification"> + This event is sent to notify the client of a custom property that + applies to this device. The property is a standard key/value store + in UTF-8 format, interpretation of both strings is left to the + client. The wayland protocol makes no guarantees about the content + of each string beyond its text encoding. + + Compositors and clients need to agree on a dictionary of properties. + For example, a compositor may designate the device to be of + 'joystick-type' 'gamepad'. This dictionary is out of the scope of + this protocol. + </description> + <arg name="property" type="string" summary="A UTF-8 encoded property name"/> + <arg name="value" type="string" summary="A UTF-8 encoded property value"/> + </event> + + <event name="done"> + <description summary="device description events sequence complete"> + This event is sent immediately to signal the end of the initial + burst of descriptive events. A client may consider the static + description of the device to be complete and finalize + initialization of the device. + </description> + </event> + + <event name="removed"> + <description summary="device removed event"> + Sent when the device has been removed from the system. + + If the client currently has the device focus, a + wp_inputfd_device.focus_out event is sent before the removed event. + See wp_inputfd_device.focus_in for more details. + + When this event is received, the client must wp_inputfd_device.destroy + the object. + </description> + </event> + + <enum name="fd_type"> + <description summary="Input fd device file descriptor types"> + This enum specifies the format of the file descriptor passed to + clients with the wp_inputfd_device.focus_in event. + </description> + <entry name="evdev" value="0" summary="An evdev file descriptor" /> + </enum> + + <event name="focus_in"> + <description summary="input fd device focus in event"> + Notification that this client now has the focus and/or access to + this device. The decision what consitutes focus left to the + compositor. For example, a compositor may tie joystick focus to the + wl_pointer focus of this seat. The protocol does not guarantee that + any specific client ever receives the focus for a device. + + The client is passed a file descriptor with access to this + device. This file descriptor is valid until a subsequent + wp_inputfd_device.focus_out event. Upon wp_inputfd_device.focus_out, the + compositor may revoke the fd and further operations will fail. + + However, due to potential race conditions a client must be able to + handle errors as if it opened the fd itself. No guarantee is + given that the wp_inputfd_device.focus_out event or wp_inputfd_device.removed + event are sent before the client encounters an error on the file + descriptor. + + A compositor guarantees that the underlying device does not change + until a wp_inputfd_device.removed event. In other words, if the fd + type allows querying capabilities through the fd, a client needs to + do so only once at the first focus_in. Subsequent focus_in events + will provide the same capabilities. + + If applicable, this event contains the surface that has the focus. + In some cases, the focus may not be tied to a specific client surface + but is given to the client independent of any surface. In that case, + the surface is null. + + The protocol guarantees that focus_in and focus_out always come in + pairs. If the client currently has the focus and the device is + removed, a focus_out event is sent to the client before the + wp_inputfd_device.removed event. + </description> + <arg name="serial" type="uint"/> + <arg name="fd" type="fd" summary="file descriptor to the device"/> + <arg name="fd_type" type="uint" enum="fd_type" summary="fd type" /> + <arg name="surface" type="object" interface="wl_surface" summary="The current surface that has the device's focus" allow-null="true"/> + </event> + + <event name="focus_out"> + <description summary="input fd device focus out event"> + Notification that this client no longer has focus and/or access to + this device. Further reads from this device's file descriptor + will fail. The client must close(2) the file descriptor received in + the wp_inputfd_device.focus_in event. + + This event does not mean the device was removed, merely that the + device is focused elsewhere. For device removal, see + wp_inputfd_device.removed. + + See wp_inputfd_device.focus_in for more details. + </description> + </event> + </interface> +</protocol> + + </blockquote></div><div class="gmail_signature"> </div> </div></div>