[PATCH libinput 0/24] Tablet support

[Peter Hutterer]

On Wed, May 28, 2014 at 10:23:00AM -0700, Jason Gerecke wrote:
> On Tue, May 27, 2014 at 8:32 PM, Chandler Paul <thatslyude at gmail.com> wrote:
> > On Wed, 2014-05-28 at 12:48 +1000, Peter Hutterer wrote:
> >> On Tue, May 27, 2014 at 03:40:07PM -0700, Jason Gerecke wrote:
> >> > On Tue, May 27, 2014 at 3:20 PM, Peter Hutterer
> >> > <peter.hutterer at who-t.net> wrote:
> >> > > On Tue, May 27, 2014 at 04:32:14PM -0400, Chandler Paul wrote:
> >> > >> On Tue, 2014-05-27 at 13:11 -0700, Jason Gerecke wrote:
> >> > >> > I've been away from my computer for most of the (long) weekend up
> >> > >> > here, so apologies for being a bit quiet :)
> >> > >>
> >> > >> > There's a subtlety on the protocol side of things that can't be
> >> > >> > ignored. When normalizing data, you want to be careful to preserve
> >> > >> > information about the zero point. Without that, you can't meaningfully
> >> > >> > pass the data along. Lets imagine that we have some sensor that will
> >> > >> > report values between 10 and 100, with a resolution of 1 unit = 1
> >> > >> > elbow per square ounce. If we normalize that to the range [0,
> >> > >> > UINT32_MAX] we've lost information about where "zero" is. A normalized
> >> > >> > value of zero does not correspond to zero elbows per square ounce as
> >> > >> > you might expect, and the resolution info is insufficient to correct
> >> > >> > the offset.
> >> > >> >
> >> > >> > Now, if we've done our jobs properly in libinput, that shouldn't be a
> >> > >> > problem. We would have normalized that sensor's values to [0.1, 1] and
> >> > >> > announced the axis to have a resolution of 1 unit = 100 elbows per
> >> > >> > square ounce. Because the zero point is offset like it originally was,
> >> > >> > it's preserved through the scaling done for the protocol and so the
> >> > >> > original 10-100 range can be recovered. The only amendment I'd make is
> >> > >> > to use a signed integer type rather than an unsigned one, since we may
> >> > >> > have negative normalized values that need to be sent through the
> >> > >> > protocol.
> >> > >> I just wrote code to normalize it to INT_MAX, but since everything's in
> >> > >> fixed point integers the actual values it's being scaled to are
> >> > >> 0-8388607.99609375 when the fixed point axis value is converted back
> >> > >> into a double, which as I'm sure you probably realize is kind of a
> >> > >> strange value, and I'm starting to think something like 0.1-1.0 would be
> >> > >> a lot better, trying to normalize to INT_MAX results in something that
> >> > >> sounds really weird to work with.
> >> > >
> >> > > we need a LI_FIXED_MAX then. Normalising to 0-1 in a 24.8 fixed point only
> >> > > leaves us with 256 value per axis.
> >> > >
> >> > Yeah, we don't want to pass a value like that through the fixed type.
> >> > It either needs to be re-scaled to use the full range (be that
> >> > [INT_MIN, INT_MAX] or [FIXED_MIN, FIXED_MAX]) or sent with a type that
> >> > won't loose quite as many bits :D
> >>
> >> didn't think of it until after I sent the previous email:
> >>
> >> there's a side-effect that we need to be aware of: if we scale an axis to
> >> LI_FIXED_MAX, we're effectively guaranteed to get 32-bit integer overflows
> >> on operations with that value. So we're effectively forcing the caller to
> >> work with int64_t to be on the safe side.
> >>
> >> which isn't the worst of all things: on fixed 24.8 with current devices
> >> overflows aren't _that_ hard to trigger so we pretty much get to pick
> >> whether they happen sometimes for some devices or all the time, effectively
> >> forcing all callers to handle this correctly from day 1.
> 
> I thought that scaling-up the range to LI_FIXED_MAX was purely an
> intermediate form that would then be scaled-back to either a
> normalized or canonical value by the wl-client. I don't know who in
> their right mind would try to use the full-range 24.8 data directly
> without first dividing it down.

remember, libinput output != what the wl client sees. the rough architecture
is:
  kernel -> libinput -> compositor |wl protocol| wl toolkit - application

so the compositor decides what's on the protocol and libinput merely adds as
a filter. the compositor can't add to what libinput provides, but it can
filter or modify values it gets from libinput.
or to compare it to the Xorg stack, think of libinput as the equivalent to
the wacom/evdev/synaptics drivers - they generate events but don't have a
say in what the clients eventually see.

so if libinput provides scaled output that may or may not be what the
clients see, but that depends on the wl protocol. having said that, I'd
probably say the wl protocol should use 24.8 normalized as well, so what you
say still applies :)

> > I've already implemented scaling to LI_FIXED_MAX in my tree, but I agree
> > this is something we should probably try to avoid. My thought here is
> > that we could scale it to a smaller value that's extremely unlikely to
> > ever overflow. My idea is to scale it to a 16.8 bit value. I don't think
> > it's likely that we'll ever be encountering anything larger then a 16
> > bit value with a wacom tablet. This does kind of seem like a bit of a
> > hack though.
> 
> There are actually a handful of tablets that exceed 16 bits for their
> X/Y axes. The Cintiq 24 for instance has a logical resolution of
> 104480x65600.
> 
> >>
> >> > >> Also, what exactly is a "zero-point" in this context?
> >> > >
> >> > > whatever the neutral state of an axis is. e.g. tilt goes in both
> >> > > directions so the effective range is -value ... 0 ... +value.
> >> > >
> >> > Or as I like to think about it, the point where an axis would report a
> >> > value of "0". Most axes will report that value, and you'll just want
> >> > to be sure that an input of 0 turns into an output of 0. There's a
> >> > corner case where 0 could be outside the range of the axis, but if you
> >> > treat it as though it could you'll end up mapping the range
> >> > appropriately (e.g. treating an axis that only reports [10, 100] as
> >> > though it really reported [0, 100] will result in proper
> >> > normalization).
> >>
> >> new question: what about a device that has an zero state that's not halfway
> >> between min/max? currently it'd advertise a range of [-N, M]. normalising
> >> those means we lose that information.
> > I don't think there's any devices like that right now, but if there are
> > it might be better just to pass the value to the client as-is without
> > normalizing it like we do with the pressure axis.
> >>
> 
> I'm confused. I don't expect axes exposed through libinput to
> advertise a range. The range is implicitly [-1, 1] because its a
> normalized value. Some hardware may never report certain values (e.g.
> an Intuos currently won't report a negative X value) but the average
> software should not particularly care about that fact.

[...]

> 
> The evdev device reports canonical values in the range [-100, 511]. A
> resolution of 1000 units/kilogram-force has been set, allowing
> applications that care about absolute forces to recover the original
> value. When libinput adds this device, it normalizes the device range
> to [-1, 1] being sure to map an input value of 0 to an output value of
> 0. Because the input range is asymmetric, some values in its implicit
> [-1, 1] range will never be sent. The Wayland protocol does not have a
> convenient "float" type that can accurately contain these values, so
> it scales the implicit input range to match the full range of some
> type (e.g. "fixed" and its [-8388608, 8388607.99609375] range). On the
> receiving end, GDK uses wl-client to scale the full-range value back
> to normalized and then clamps it to the range [0, 1] since that is the
> documented range of GDK_AXIS_PRESSURE. GIMP is notified by GDK of the
> pressure and uses it as a scaling factor on the brush size to achieve
> some size between zero and 8 pixels.

yeah, I guess that's what I'm not a big fan of: reporting a range that the
device cannot actually achieve. If you switch your example around and you
have a device that goes from [-511, 100], GIMP will never scale up to the 8
pixels, despite the user using the maximum pressure on the device.

Which may not matter after all, maybe no-one cares about this anyway. It
would be easy enough to add at a later point to say that effectively,
min/max is bounded by $whatever, not by the normalized range.
but at that point we're now providing exactly what the kernel gives us, but
mangled into a different format.

> GIMP (and the majority of desktop applications) do not care about the
> absolute range of the input data because they only use the normalized
> value as a scaling factor. If we made a slightly different assumption
> about how our device above works (lets say it has a range of [100,
> 511] instead of [-100, 511]) it doesn't change the picture for GIMP:
> 50% of the maximum force will normalize to a value of 0.5 either way,
> and will result in a brush 4px in diameter. Its possible that an
> application which pays attention to units could make use of range
> information, but only insofar as letting the user know that some
> values are beyond the sensor's capabilities.

..and we're also assuming that 0 actually means zero/neutral on a device,
and devices never have a truly arbitray axis range of [-N, M] with the
neutral point at 0 + X.

Cheers,
   Peter