Questions and thoughts about input method protocol

[Yichao Yu]

Hi,

With the comment on the recent patches for the input method protocol,
it seems that we are finally on the way to support cursor following.
However I still have some questions about the protocol (and it's
limitation).

1, Is there any plan to support xwayland?

I believe it is important to support using input method in x clients
running on xwayland. IMHO, the input method can still use xim or it's
own protocol to get key event from and send input result (as well as
preedit etc.) to the client running on xwayland with no problem.
However, I cannot see a perfect solution to make cursor following
works using the proposed way to locate the input overlay surface.

For application using xim, maybe it is possible to let xwayland handle
the events and then forward them to the text-model. However, first of
all, this cannot work for x client talking with input method using a
private protocol and it will be a HUUUUGE regression if we force all x
clients to use the broken xim (application frozen, wrong support of
cursor following and preedit etc.). Moreover, since xim support is so
different in different applications, the input method sometimes have
to do some hack on xim, which will not very likely be something we
want to add in xwayland.

(Maybe adding some interface to interact with xwayland to set cursor
position on certain x window surfaces?)


2, Is it possible for the input method to know anything about the client?

Some famous (Chinese) input methods (on Mac and Windows) support the
so-called context awareness, in another word, the input method will
use some information of the client to determine the candidate words
list (and its order). This may not be that useful for Latin based
languages (although it may also be good if you want to provide
spelling hints) but if your are facing a language with 3000-5000
frequently used characters and frequently used words 10-100 times of
this number depending on your context, this shouldn't be ridicules at
all.

Currently, although I don't know any input method on linux support
context awareness, it is possible to do this under X since all the
necessary information is accessible to the input method. These include
all general information the window system and a plugin running in the
client's process can know, including (most important and useful)
window titles (with other WM related properties like icon names,
application names etc.), pid's (plus host's for X) and window id's
etc. The pid's and window id's are also very useful for getting more
information from the underlying system (/proc for example) about the
client (e.g. command line arguments) and can also be used to provide
per program or per window input state for some programs (Fcitx support
both.)

Right now I don't think there is any way to get these information from
the input method protocol. It will be a big regression (not as big as
not supporting cursor following in x clients though) if this cannot be
supported in wayland.

NOTE: The "context type" added in the recent patches may also be
helpful on this but they are different. It is indeed helpful for input
method to know the user is typing in a url/search bar instead of a
normal text entry but the stuff you may want to search may be very
different on amazon and arxiv.


3, Is it possible to let the user move the keyboard panel while
keeping it off from the cursor position when possible?

Maybe it is already possible and I just don't fully understand the
input_panel interface but anyway I guess this should be supported,
either by the compositor or make it possible to let input methods
support it.


4, For running different component of the input method in different processes.

>From the updated doc string of input_panel, you are saying it is only
possible for one client to bind to the interface. Well, IMHO, I don't
really think this is necessary.

It will be indeed a problem if two input methods both want to handle
key events on the same seat. This can be easily avoided by only
allowing one client binding to the input_method interface. The input
method can just quit/disable wl support/retry later/do what ever it
want if that bind fails.

However, for the input_panel interface, I don't really think this is
necessary. For conflict with another input method, this should be
already handled by the input_method interface. Also this can increase
the flexibility of the input method by allowing virtual keyboard and
input overlay window drawing by two different process (not only just
in a single process although different with the one binding to the
input_method interface which get key events).


5, For the proposed input_panel_surface::cursor_position event.

I guess it will be better to change it to a cursor_rect event which
also includes the size of the cursor. No?


6, Some random stuff of the current interface.

There seems to be a password context type. I think normally a password
field will not have input context or is that for using virtual
keyboard in password field?

There seems to be a empty text_model::set_preedit request for the
client. Shouldn't this be fully controlled by the input method?? (Plus
there isn't a corresponding event on the input method side.... anyway
it's just weird for me....)


Yours,
Yichao Yu