Questions and thoughts about input method protocol

[Pekka Vuorela]

On 29.01.2013 04:49, Yichao Yu wrote:
> Hi,
>
> With the comment on the recent patches for the input method protocol,
> it seems that we are finally on the way to support cursor following.
> However I still have some questions about the protocol (and it's
> limitation).
>
> 1, Is there any plan to support xwayland?
>
> I believe it is important to support using input method in x clients
> running on xwayland. IMHO, the input method can still use xim or it's
> own protocol to get key event from and send input result (as well as
> preedit etc.) to the client running on xwayland with no problem.
> However, I cannot see a perfect solution to make cursor following
> works using the proposed way to locate the input overlay surface.
>
> For application using xim, maybe it is possible to let xwayland handle
> the events and then forward them to the text-model. However, first of
> all, this cannot work for x client talking with input method using a
> private protocol and it will be a HUUUUGE regression if we force all x
> clients to use the broken xim (application frozen, wrong support of
> cursor following and preedit etc.). Moreover, since xim support is so
> different in different applications, the input method sometimes have
> to do some hack on xim, which will not very likely be something we
> want to add in xwayland.
>
> (Maybe adding some interface to interact with xwayland to set cursor
> position on certain x window surfaces?)

X based input methods are using X windowing, so wouldn't they just 
continue working for X apps? (Provided that xwayland has the ability to 
put windows into absolute positions). Translation between wayland text 
<-> XIM doesn't seem worthwhile to me.


> 2, Is it possible for the input method to know anything about the client?
>
> Some famous (Chinese) input methods (on Mac and Windows) support the
> so-called context awareness, in another word, the input method will
> use some information of the client to determine the candidate words
> list (and its order). This may not be that useful for Latin based
> languages (although it may also be good if you want to provide
> spelling hints) but if your are facing a language with 3000-5000
> frequently used characters and frequently used words 10-100 times of
> this number depending on your context, this shouldn't be ridicules at
> all.
>
> Currently, although I don't know any input method on linux support
> context awareness, it is possible to do this under X since all the
> necessary information is accessible to the input method. These include
> all general information the window system and a plugin running in the
> client's process can know, including (most important and useful)
> window titles (with other WM related properties like icon names,
> application names etc.), pid's (plus host's for X) and window id's
> etc. The pid's and window id's are also very useful for getting more
> information from the underlying system (/proc for example) about the
> client (e.g. command line arguments) and can also be used to provide
> per program or per window input state for some programs (Fcitx support
> both.)
>
> Right now I don't think there is any way to get these information from
> the input method protocol. It will be a big regression (not as big as
> not supporting cursor following in x clients though) if this cannot be
> supported in wayland.
>
> NOTE: The "context type" added in the recent patches may also be
> helpful on this but they are different. It is indeed helpful for input
> method to know the user is typing in a url/search bar instead of a
> normal text entry but the stuff you may want to search may be very
> different on amazon and arxiv.

Not wanting to ridicule chinese input needs, but fetching window title, 
executable name and/or application parameters to alter behavior is a 
_huge_ hack. Going deeper into knowing what url is visible in a browser 
is even worse. Does not take much effort to break that kind of 
functionality.

I would try to find better ways to achieve targeted goals. One feature 
I've myself been thinking is virtual keyboard used with messaging app 
knowing what language, or type of language, the recipient is speaking 
and adjusting prediction/layout based on that. That could be done by 
having locale in text editor's state, but alternatively could also be 
editor setting a globally unique identifier (integer, string?) as state 
for text fields, after which text input side could learn what language 
got used. Similarly for chinese input such mechanism could be used to 
remember preferences. Out-of-the box amazon and arxiv wouldn't make 
difference, but after some use they would. To support this, the toolkits 
and applications would also need some enhancements, though.


> 6, Some random stuff of the current interface.
>
> There seems to be a password context type. I think normally a password
> field will not have input context or is that for using virtual
> keyboard in password field?

All text fields, including password ones, should generally be using 
input methods. Virtual keyboard is one case, but there could be also 
some other kind of composing even for hardware keyboard. Think of long 
press to get different characters etc.

> There seems to be a empty text_model::set_preedit request for the
> client. Shouldn't this be fully controlled by the input method?? (Plus
> there isn't a corresponding event on the input method side.... anyway
> it's just weird for me....)

Was discussed last October:
http://lists.freedesktop.org/archives/wayland-devel/2012-October/005866.html

I suggested and Jan Arne agreed on removing it. Pending work, I suppose.