Questions and thoughts about input method protocol

[Yichao Yu]

On Fri, Feb 1, 2013 at 2:51 AM, Jan Arne Petersen
<jpetersen at openismus.com> wrote:
> On 29.01.2013 03:49, Yichao Yu wrote:
>> With the comment on the recent patches for the input method protocol,
>> it seems that we are finally on the way to support cursor following.
>> However I still have some questions about the protocol (and it's
>> limitation).
>>
>> 1, Is there any plan to support xwayland?
>
> I do not have a plan for that, but it should be possible to add some
> protocol so that an input method could tell about a X window and the
> cursor position (which it got over some legacy X input method protocol)
> and the input panel surface would than be placed correctly on that
> xwayland surface.

It will be indeed a lot better if the input method don't have to
create a xwayland surface (X window) for the input panel rather than a
wayland input panel surface.

>
>> 2, Is it possible for the input method to know anything about the client?
>>
>> Some famous (Chinese) input methods (on Mac and Windows) support the
>> so-called context awareness, in another word, the input method will
>> use some information of the client to determine the candidate words
>> list (and its order). This may not be that useful for Latin based
>> languages (although it may also be good if you want to provide
>> spelling hints) but if your are facing a language with 3000-5000
>> frequently used characters and frequently used words 10-100 times of
>> this number depending on your context, this shouldn't be ridicules at
>> all.
>>
>> Currently, although I don't know any input method on linux support
>> context awareness, it is possible to do this under X since all the
>> necessary information is accessible to the input method. These include
>> all general information the window system and a plugin running in the
>> client's process can know, including (most important and useful)
>> window titles (with other WM related properties like icon names,
>> application names etc.), pid's (plus host's for X) and window id's
>> etc. The pid's and window id's are also very useful for getting more
>> information from the underlying system (/proc for example) about the
>> client (e.g. command line arguments) and can also be used to provide
>> per program or per window input state for some programs (Fcitx support
>> both.)
>>
>> Right now I don't think there is any way to get these information from
>> the input method protocol. It will be a big regression (not as big as
>> not supporting cursor following in x clients though) if this cannot be
>> supported in wayland.
>>
>> NOTE: The "context type" added in the recent patches may also be
>> helpful on this but they are different. It is indeed helpful for input
>> method to know the user is typing in a url/search bar instead of a
>> normal text entry but the stuff you may want to search may be very
>> different on amazon and arxiv.
>
> There are two use-cases here:
> If you want to add some kind of manual mapping between application and
> candidate suggestions used, I would rather add explicit ways for
> applications/websites to better specify the content/context of an input
> entry.

It's not a direct map, rather a hint. Some protocol like a
context_string and a url will be good. Having that on top of a
key-value protocol can give better flexibility.

To clarify what a input method will need and why, here is a short list
of them with explanation.
1. pid: this is already used now to provide per-program input state.
(It's not related to context aware input and isn't a hack at all. I
put it here mainly because it is something a input method needs to
know that is not yet possible.)
2. program name: something that can be directly know from that which
may be useful is categories of the program from the desktop file.
3. context string(s): in most cases, the window title will be really
good enough. For a browser, this may be the title of the webpage, url,
or other info from the page describing what the page is all about.

> For candidate suggestions based on previous usage patterns (by the same
> user or other users via some remote database) an UID approach as
> mentioned by Pekka would be useful.

Just in case it is still not clear enough, that is not the right
solution. The only right way to do is have a one-to-one corresponding
between text_model and input_method_context so that the input method
is able to track any state it want.

>
>> 3, Is it possible to let the user move the keyboard panel while
>> keeping it off from the cursor position when possible?
>>
>> Maybe it is already possible and I just don't fully understand the
>> input_panel interface but anyway I guess this should be supported,
>> either by the compositor or make it possible to let input methods
>> support it.
>
> There is currently no support for that, but it should be possible to add
> support for that.
>
>> 4, For running different component of the input method in different processes.
>>
>>>From the updated doc string of input_panel, you are saying it is only
>> possible for one client to bind to the interface. Well, IMHO, I don't
>> really think this is necessary.
>>
>> It will be indeed a problem if two input methods both want to handle
>> key events on the same seat. This can be easily avoided by only
>> allowing one client binding to the input_method interface. The input
>> method can just quit/disable wl support/retry later/do what ever it
>> want if that bind fails.
>>
>> However, for the input_panel interface, I don't really think this is
>> necessary. For conflict with another input method, this should be
>> already handled by the input_method interface. Also this can increase
>> the flexibility of the input method by allowing virtual keyboard and
>> input overlay window drawing by two different process (not only just
>> in a single process although different with the one binding to the
>> input_method interface which get key events).
>
> To the input_panel interface will be bound only by the shell process
> (like weston-desktop-shell). input_panel_surfaces can be created by
> multiple processes.

Isn't the input_panel interface used to create / set a surface as a
input_panel_surface ?

>
>> 5, For the proposed input_panel_surface::cursor_position event.
>>
>> I guess it will be better to change it to a cursor_rect event which
>> also includes the size of the cursor. No?
>
> Yes, that would make more sense.
>
>> 6, Some random stuff of the current interface.
>>
>> There seems to be a password context type. I think normally a password
>> field will not have input context or is that for using virtual
>> keyboard in password field?
>
> Pekka already answered that.
>
>> There seems to be a empty text_model::set_preedit request for the
>> client. Shouldn't this be fully controlled by the input method?? (Plus
>> there isn't a corresponding event on the input method side.... anyway
>> it's just weird for me....)
>
> Yes sorry, I removed it in one of the cleanup commits.

=).

>
> --
> Jan Arne Petersen
> Openismus GmbH
> http://www.openismus.com
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel