[PATCH libinput 0/24] Tablet support

[Peter Hutterer]

On Thu, May 22, 2014 at 03:37:44PM -0400, Chandler Paul wrote:
> On Thu, 2014-05-22 at 11:17 -0700, Jason Gerecke wrote:
> > On Wed, May 21, 2014 at 10:17 PM, Chandler Paul <thatslyude at gmail.com> wrote:
> > > Hi! Sorry this took so long to write, I've been spending a lot of my
> > > time recently trying to understand the libinput code and all of that
> > > good stuff, and I wanted to make sure I had a decent understanding of it
> > > before I actually wrote up a response.
> > >
> > I'll take a look over the patches and see if I can provide any
> > comments. I haven't looked at the libinput code before, so I'll
> > probably be stumbling around for a bit :D
> 	That's fine, I'm very thankful you're taking the time to help me out
> anyway :). Plus I'm still stumbling a bit on libinput myself, so I can
> relate!
> 
> > IMO, normalization makes a lot of sense and libinput is a fine place to do it.
> > 
> > Applications use axis data for various reasons. For instance an app
> > may read pressure values to adjust brush size, change the ink flow
> > rate, or possibly even simulate the pressure-varying effects of a
> > paint knife. When using data, the application can interpret it in two
> > ways*: as unitless quantity or as a number with some attached unit.
> > _Many_ applications don't particularly care about units and will
> > simply ignore any unit information provided. They take the data,
> > normalize it, and use that fractional value to scale some other
> > quantity (brush size, flow rate, degree of paint drag). Even data that
> > you normally associate with a unit is typically treated this way. For
> > instance, tilt angles are liberally interpreted and as long as the
> > values are centered and increase/decrease properly most applications
> > could care less about the underlying units**. For these applications,
> > normalizing the data up-front just makes things easier: the API is
> > cleaner since you just move around a bunch of floats, and it saves
> > applications from doing the normalization that they're going to do
> > anyway.
> 	You're completely right on this. Even if we provided functions for
> normalizing the data, having our default behavior with returning axis
> values aimed at a niche market of applications probably isn't the best
> idea, and will leave implementations lot messier then we want them. The
> more functions a client needs to call, the messier and less appealing
> working with libinput will be.

as Jasper said in the other email, the clients are the wayland compositors
and xf86-input-libinput (and a few debugging tools). By the time a wayland
client or X client gets the data, it's been filtered through the protocol.

Which means that the discussion is still valid, but applies to the wayland
protocol more so than libinput. And it means that whatever libinput filters
or otherwise mangles can't be sent over the protocol.

> > Now, that said, there is a small class of applications that make use
> > of the units for one reason or another. Perhaps it is a drafting
> > application that measures physical distances in cm between two points,
> > or perhaps it is that drawing application from earlier now modeling
> > the physical effects of a given number of newtons of force on the
> > tool/ink/canvas. These kinds of uses are niche, but we should
> > absolutely not prevent developers from being able to make them. What
> > is important to the developer in this case is that the data is
> > accompanied with enough information to put it into physical context;
> > at the very least this means transmitting the physical resolution of
> > the data. Now, you could provide the raw data to an application but
> > there's not really any point. One application might want to work in
> > centimeters, the next in inches, the next in meters. In short, though
> > applications interested in units aren't going to /normalize/ the data,
> > there's a good chance they're going to /rescale/ the data. And if
> > they're rescaling the data to their unit-of-choice anyway, it doesn't
> > really matter what unit the data is ingested in since the resolution
> > information is available.
> > 
> > Based on that, I think it makes a lot of sense to pre-process the
> > data. Applications don't use the raw values as-is anyway, so if
> > pre-processing allows for a simpler and easier-to-understand interface
> > then it should be seriously considered. Some axes might make more
> > sense canonicalized rather than normalized (e.g. angles might make
> > more sense in radians rather than fractions of a [semi-]circle) but
> > that's a separate issue.
> 	You bring up a very good point, the way you put it normalization
> definitely makes a lot of sense, and I probably will make sure it can be
> done through libinput. And you're right; libraries are meant to give
> developers the ability to do things in their applications more easily,
> and if we're limiting the values returned to being normalized or being
> raw we're going against that philosophy. I see uses for raw values, but
> with the points you brought up I also definitely see uses for normalized
> values too. So I think I'm going to re-implement normalization on my
> branch in a way that allows a client application to do both.
> 
> 	he more functions a client has to call to get the values they need, the
> messier their code will be and the less appealing using libinput will
> seem. We also want a design that will (hopefully) last for a long time,
> without needing to be completely rewritten in the future.
> 
> 	So I think the best way of doing this is to just have libinput provide
> calls for grabbing normalized values off any given axis event, and also
> provide calls for grabbing raw values off any given axis event. The
> event structures should, by default, only have fields containing the raw
> axis values. Since a client using libinput can't access the fields
> directly and can only do so through functions, they won't need to be
> concerned with this aspect. By doing this, we prevent wasting time
> normalizing values for applications that don't want it.
> 
> 	From what you've told me, it sounds like more applications will want
> normalized values then not. So by default;
> 
>         libinput_event_pointer_get_axis_value()
>         
> should return a normalized value for each tablet axis. For getting raw
> values, we should add a function to the libinput API that looks
> something like this:
> 
>         libinput_event_pointer_get_raw_axis_value()
> 
> 	Although if there's a function name that would fit better with the
> coding style of libinput, feel free to let me know.
> 
> 	This function should return the input_absinfo as-is to the client. From
> there it's their responsibility to make the value useful.
> 
> 	Just an idea for the future: It might also be cool to provide functions
> to get the axis values in centimeters, inches, etc. but I'm not sure on
> how much use this would actually be, and if only one or two applications
> ever end up using functions like that it'd probably be better to keep
> them out of libinput and leave this sort of thing to the clients.

I'm almost sold on normalization since it does reduce the likelihood of
things going wrong. We need to provide the axis resolution to convert back
to the real data though where needed.

once you provide the axis resolution, it doesn't matter if you provide raw
data unless you also want to provide "raw" resolution, which is excessive.

so, given that this would be sent down the protocol (and for the limited
resolution) the range should be normalised to uint16_t or uint32_t max, with
the resolution in units/mm or canonicalized where more appropriate. This
would be what goes on the wayland protocol as well then.

helper functions to convert that back to doubles, or elbows per square ounce
would be part of the wl-client package that parses that protcol, not
libinput.

that doesn't seem like the worst architecture for both libinput and the
protocol, any comments?

> > * There is a third vanishingly small class of applications that
> > neither normalize their data nor care about the unit of the data.
> > They, for whatever reason, need to know the underlying raw values.
> > This could be for something like a hardware debug tool or simulator. I
> > don't think this class of applications is really libinput's target. If
> > an application really-truly-absolutely needs to know the raw values,
> > it should use a low-level interface like evdev.
> 	Yes, for a lot of applications like that it probably would be better
> just to use evdev. However, since Carlos already wrote some stuff for
> normalization I think having the ability to provide both would be very
> easy to implement, and would save a lot of work on the client's part
> since they won't have to fall back on reinventing the wheel for
> everything that libinput can already handle the same way in the event
> that they do need raw values.
> 
> > 
> > ** I'd argue this is the case given that both the Qt and GTK
> > frameworks have problems with tilt angles. The former specifies data
> > to be in degrees but doesn't convert non-degree data, while the latter
> > provides no guarantee of units at all and in practice applications
> > like GIMP wind up computing the wrong physical angles.
> > 
> > > I've forked libinput and I have a branch where I'm fixing up the patches
> > > Carlos sent in based on the feedback from Peter. You can find it here:
> > >
> > >         https://github.com/Lyude1337/libinput/tree/carlos_cleanup
> > >
> > > The history is messy on this, but once this is ready to get sent in as
> > > actual patches I'll be rebasing the history.
> > >
> > > Right now I've removed normalization from my branch, but if someone
> > > brings up a good reason to actually have libinput handle that then I can
> > > revert the change.
> > >
> > > Another thing I'm considering regarding this is just having libinput
> > > provide functions/macros for normalizing the values so applications can
> > > normalize the values easily if they want.
> > >
> > That could be an interesting approach as well. After Qt and GTK both
> > screwed up tilt, I don't trust clients as far as I can throw them.
> > There should be some way to get normalized/canonical data, whether
> > that's the default (which I'd currently argue), or through some
> > conversion functions.
> 	Hopefully in the future I can change this. This kind of makes me think
> though, maybe GTK and Qt should grant the ability to easily leave the
> handling of libinput up to the client and not to the toolkit/framework.
> If I get the time I'll definitely look into this.

just to re-iterate for those skipping to the non-existent TL;DR section :)

gtk/qt won't see libinput, the stack looks like this:

kernel -> libevdev -> libinput -> weston -> wl_protocol -> Qt/GTK -> app

Cheers,
   Peter

 > 
> > > The way I see it is that we want to allow all of the axes to be used how
> > > they were meant used. If Wacom isn't normalizing the values in their
> > > in-house applications I don't think we should either. Again though, I'm
> > > curious to hear Ping and Jason's take on the reasoning for their tablets
> > > sending it's data like this.
> > >
> > > Cheers,
> > >         Lyude
> > >         (a.k.a. Stephen Chandler Paul)
> > >
> > >> * No filtering/hysteresis of coords is done yet.
> > >>
> > >> Cheers,
> > >>   Carlos
> > >>