[RFC] Device hierarchy for MPX

[Peter Hutterer]

This email is about introducing a two-layer hierarchy into the device
system for MPX [1]. It is a long email, but please read it and comment
or be silent forever.

===== Problem =====
Right now MPX gives each pointer device a separate cursor and each
keyboard device a separate keyboard focus. This is cool if you are an
octopus, but a problem if you need merged devices (e.g. touchpad and
mouse controlling the same pointer). MPX has a SharedPointer option for
devices, but I haven't tested it for ages and the grab semantics break
down on it anyway.

===== Acknowledgements =====
The problem was pointed out by Keith Packard, and this pretty much
represents what he proposed as solution.

===== Solution =====
This cursor and keyboard focus is the user’s centre of attention, not
the devices. X has an implicit two-layer hierarchy: the lower layer
representing the physical devices and the higher layer representing the
user’s control points.

If we make this hierarchy explicit, we get the notion of master and
slave devices [2].
A master device (MD) is a virtual device added by the X server
internally. It is not connected to a physical device, but has a sprite
and a focus. Multiple master devices means multiple pointers/foci. MDs
cannot be attached to other master devices. SendCoreEvents is implicit
for MDs.

Slave devices (SD) are the actual physical devices. SD's attached to an
MD control the MD's cursor/focus and send core events. SD's can be
"floating" (not attached to an MD), but then do not have a sprite and do
not send core events. An MD can have multiple SDs attached. The MD
always represents the merged state of all attached SDs. SD devices are
invisible to clients using the core protocol and do not send core events
directly.

A client can listen to events from MDs and SDs.

Note that implementation of this hierarchy breaks the input driver ABI
and XInput and thus all applications using XI (== the GIMP). Tough luck,
welcome to the $CENTURY.

===== Defaults and configuration =====
By default, the server starts up with exactly one MD. SDs as added to
the layout are added to the first MD. Only 1337 h4xx0rs will notice that
MPX is even there, average users can live simple and happy lives where
it rains flowers every day.

Configuration options will be provided to attach devices to a MD. The
first idea was:

Section "InputDevice"
	Identifier "master1"
	Option "MasterDevice" True
	...
EndSection

Section "InputDevice"
	Option "AttachToMaster" "master1"
	# Option "Floating" True
EndSection

For MDs, the "driver" option is ignored.
"Floating" and "AttachToMaster" are mutually exclusive, the server will
stab you in the pinkie if you use both together.

===== Flexible Attachment =====

MDs can be added and removed. The first (initial) MD can be removed once
a second MD is added, but at least one MD must be present at all times.

Adding MD's causes new cursor sprites to appear and adds a new keyboard
focus, removing them undisplays the cursor again (and removes the kbd
focus). Adding/removing MDs sends DevicePresenceNotify events.
An SD's attachment can be changed at runtime. If done so, clients
receive an DeviceStateChangeNotify [3] from the MD and from the SD. If
an SD's attachment is changed while a button is pressed, the SD will
send the necessary release events, and - after reattaching to the other
device - the required press events.
Adding/removing SDs sends DPN events.

"Flowchart" for moving SD1 with pressed key from MD1 to MD2

move request
- SD1 key release
- MD1 key release (depends on merged state, see below)
- SD1 DeviceStateChangeNotify (details: moving)
- MD1 DeviceStateChangeNotify (details: SD1 removed)
- MD2 DeviceStateChangeNotify (details: SD1 added)
- SD1 DeviceStateChangeNotify (details: added to MD2)
- SD1 key press
- MD2 key press (depends on merged state, see below)
move completed

The "moving" detail makes IMHO more sense than setting the SD to
"floating" and then to "added". Reduces pain on the client-side.

Changing master devices will be exposed through an ChangeDeviceControl
struct.
[The current ChangePointerKeyboardPairing in MPX will be removed].

===== Merged states =====
The merged state from the MD follows effectively the same rules as the
current behaviour of the core devices. E.g. if mouse1 and mouse2 press
button 1 and 2 respectively, the MD has buttons 1 & 2 set.

Button remapping takes in affect _before_ the MD sets state. If SD swaps
buttons 1 and 2, pressing physical button 2 on the SD causes a button 1
press on the MD.

[Daniel, this should work for keyboards as well, right?]

It gets a bit more difficult when we're talking about axis mapping.
Imagine the case where we have a usb mouse and a 3D absolute input
device attached to an MD. The mouse sends relative coordinates, clipped
to the screen. The other device can send device coordinates not
necessarily related to screen coordinates. Mapping them to one
consistent notation loses information.

The MD thus takes on the identity of the currently active device. So
after the mouse moves, the MD has 2 relative valuators. If the 3D device
moves, the MD changes classes and now has 3 absolute valuators. Before
the event is sent, a DeviceStateChangedNotify is sent to inform the
client. The client has to be able to handle the different coordinate
systems. Oh how much fun client-side programming will be in the future.

Querying input devices (XListInputDevices) will only reveal the
_current_ state of an MD, you need to listen to the DSCN events.

Setting the device focus for an MD sets the focus for the SD. Setting
the focus for a non-floating SD sets the focus for the MD as well.

===== Grabs =====

An MD grab blocks grabs on all attached SDs (AlreadyGrabbed).
An MD can be separately grabbed with a pointer and a keyboard grab

An SD grab temporary removes the SD from the MD (DSCN events being sent
from SD and MD with a field set to "grab"). Events during the SD grab
are _not_ reported through the MD, and the SD is detached from the
sprite. Changes to the MD (e.g. focus changes) while the grab is active
does not change the SD.
Releasing the grab adds the SD to the original MD again.

The SD will add itself to the first MD if the original MD was removed
during the grab.

A MD grab while a SD is detached is possible. Re-attaching the SD will
add the SD to the MD's grab.

Grab semantics in between MDs stay at the current MPX semantics [4]

Reason for the detachment is that a grab should guarantee unique access
to a device. If multiple devices move the same sprite, everything gets
really difficult.
[yes keith, you were right, this is the only sensible option. it will be
"fun" to implement.]

I think this is enough for now. More details come as necessary.

Cheers,
    Peter

[1] http://wearables.unisa.edu.au/mpx/
[2] Politically correct naming suggestions welcome.
[3] New event being introduced. Contains all the information about the
device's current state, including attachment, axis, buttons, etc.
[4] http://wiki.x.org/wiki/Development/Documentation/MPX