VMware & Wayland [WAS Re: Blender3D & cursor clamping]

[Stefanos A.]

As a game developer who recently had to implement mouse/keyboard input for
Windows, Linux (core and XI2) and Mac OS X, let me try to describe how an
ideal input system would work. This will be slightly gaming-centric but I
believe the following also covers VMWare's use case completely.

There's some brainstorming going on here, so please bear with me.

1. Absolute vs relative motion

Most games require either relative or absolute motion events. The former are
generally used in 3d games with 360 degree cameras (e.g. first-person
shooters) that require movement unbound by the window size. The latter are
generally used in 2d or top-down games (e.g. real-time strategy) where the
pointer always corresponds to a specific on-screen pixel.

A game may require seamless changes from absolute to relative motion or vice
versa at any point. For instance, first-person shooters (relative) also have
menus (absolute), whereas strategy games (absolute) also have grab&pan modes
(relative).

A few, more uncommon, games may also require absolute and relative events at
the same time. Whereas independent absolute/relative modes can be emulated
even on systems without such support (e.g. core X11), the latter mode
*cannot* be implemented without explicit support. This is why the Wine
project encounters issues emulating specific Windows games: core X11 only
supports absolute events, while Windows supports both absolute and relative.
(XInput2 solves this issue, but more on that later).

In short, we need a way to get both the current location of the pointer on
screen *and* the relative raw motion (regardless of whether the pointer
actually moved or not).

Windows has two distinct input systems: one absolute and one
relative/absolute depending on the device (raw input mouse vs tablet). Core
X11 only supports absolute events. XInput2 provides both relative and
absolute events, as does Mac OS X.

2. Raw vs scaled input

Absolute events should always be scaled by the acceleration curve of the
server. Relative events should be reported both with and without
acceleration applied.

XInput2 gets this 100% right. Windows only provide unscaled raw events but
allow you to apply the acceleration curve manually (although the API is
rather hairy). As far as I can tell, Mac OS X only provides scaled values,
which is a disadvantage for games and applications that apply their own
acceleration curves (like the guest OS in VMWare).

3. Constrained vs unconstrained input

Applications should have a way to indicate that they don't want the pointer
to leave a specific rectangular region. The pointer will stop moving when it
hits a boundary of the region (but relative events will continue to be
generated).

This is useful for games and applications that require both absolute and
relative events (e.g. strategy games or VMWare with a cursor grab).

4. Mouse warping

This is an important accessibility feature and is provided by all operating
systems. The question is whether the waring function should generate motion
events.

The X protocol requires that motion events be reported as if the user had
moved the mouse. This feature is ill-designed and has led to untold numbers
of hacks. Three solutions: provide an explicit way to suppress those events;
set a property to indicate that the motion event is synthesized; or generate
absolute events but do not generate relative events.

Windows follows the second solution. Mac OS X follows the third solution.
The XInput2 people are considering the first solution. My vote goes for the
second: applications keep working without any explicit change and any
application that needs to suppress synthesized events can do so.

5. Mouse tracking

Applications should be able to receive motion events even when the pointer
leaves the application window if desired. This feature should be disabled by
default and enabled by the application under specific circumstances (e.g.
drag&drop motions).

6. Cursor grabs

On X11, cursor grabs are (ab)used to implement features #1, #3 and #5. With
explicit support for those features, cursor grabs lose their value - which
is a great thing, considering that grabs have extremely annoying side
effects when an application hangs (user cannot click away) or when you are
debugging an application (your mouse becomes useless once you hit a
breakpoint).

A grab mode might be desirable for password entries. In that case, it should
really be called "password mode", whereupon all input events are redirected
to the requesting window. Applications with elevated permissions (like the
WM) should be able to break out of a password mode but applications with
lower or equal permissions should not.

In any case, it should be made very clear that password mode should be used
sparingly and with great care.

7. Foreground vs background windows

Background windows should be able to receive input events if explicitly
requested but should not be able to constrain the cursor or initiate
password mode. It might be desirable to allow mouse warping to work for
accessibility reasons.

When a foreground window moves to the background, its constraining region
should be disabled. A foreground window in password mode should remain
uninterruptible unless explicitly requested by an elevated process (for
instance: the user presses 'alt-tab' or 'minimize' and instructs the
(elevated) WM to interrupt password mode).

Any other mutable input settings (like acceleration curves or screen
resolution) should be application-specific and the server should restore
them once the application loses focus or exits. Permanent changes should
only be possible through elevated applications.

8. Multiple pointers

By default, pointer events should be reported as the aggregate of all
individual pointing devices. Applications should be able to request
reporting of individual events via a simple API. In that case, moving an
individual device would generate an event for both that device and the
aggregate (virtual / master) device.

XInput2 has an elaborate master/slave system, where multiple slave devices
are attached to master devices and events can be selected for either. This
is the most flexible solution but whether this kind of flexibility is
actually useful is another matter entirely. Maybe YAGNI is the best approach
here: provide only what cannot be implemented directly in applications
(multiple master devices *can* be implemented client-side without too much
work. Multiple individual devices, however, require server support).

9. Mouse wheel(s)

For the love of god, mouse wheels are *axes* not *buttons*. Both core X11
and XInput2 get this completely wrong and expose wheels as buttons. This is
evil, this is wrong and this makes wheel acceleration impossible to
implement in any feasible way. It also makes reliable mouse polling
impossible on X11 (since wheel button events have a duration of 0).

To reiterate, wheels should be treated as additional, stateful axes (z, w,
...). Their absolute and relative values should be reported in events with
acceleration curves should be applied just like in the x and y axes (and
feature #2).

10. Acceleration curves

These probably belong to the server, in order to ensure consistency. It
should be possible to specify different curves for each axis. Applications
should be able to modify those curves for while they are alive and in focus.
Only elevated applications should be able to modify those globally.


This is all I can think of right now. Feel free to criticize and/or flame,
I'll might be able to help with all this once I can get Wayland running in
VirtualBox.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-devel/attachments/20101214/4c8de8c6/attachment-0001.html>