Input method and input/output redirect under wayland

[Yichao Yu]

On Thu, Aug 30, 2012 at 5:56 PM, Jan Arne Petersen
<jpetersen at openismus.com> wrote:
> On 08/30/2012 11:15 PM, Yichao Yu wrote:
>> On Thu, Aug 30, 2012 at 3:32 PM, Jan Arne Petersen
>> <jpetersen at openismus.com> wrote:
>>> The im server would just specify a surface, see current input_panel
>>> interface at
>>> http://cgit.freedesktop.org/wayland/weston/tree/protocol/desktop-shell.xml?id=86adef9d54d41b1594749c00abab24482248fa5c#n102.
>>> I plan to merge that interface into the input_method interface though
>>> and there will be an enum argument for having the surface positioned at
>>> the cursor (currently it just supports virtual keyboard like surfaces at
>>> the bottom of the screen).
>> Following the cursor definitely need to be supported.
>>
>>>
>>> The application will notify the compositor about the cursor location via
>>> the set_micro_focus request in the text_model interface
>>> http://cgit.freedesktop.org/wayland/weston/tree/protocol/text.xml?id=86adef9d54d41b1594749c00abab24482248fa5c#n20
>>
>> So if I understand it correctly (correct me if I am wrong), the im
>> server can get information about the input surface (e.g. sth like a
>> text entry widget) and the cursor location in local coordinate of that
>> surface (which is not redirected) and it can request the compositor to
>> locate (and redirect if necessary) its overlay window at/around that
>> position?
>
> The im server would not get the cursor location itself but just request
> to get its overlay window positioned at the cursor.
>

What about the relative position to the cursor?

For example, the overlay window might contain both a "preedit" and a
vertical candidate list (which is pretty common) and suppose the input
surface is close to the edge of the screen. Obviously, we want to put
the overlay window inside the screen (if possible). So who will be in
charge of the window positioning?

AFAIK, the im server (as a wayland client) cannot do this because it
has no idea where the edge of the screen is, and if we let the
compositor position the window, the im server will not be able to set
the orientation of the overlay window accordingly. For example, when
it is closed to the bottom[1] or not[2]. This is actually the simplest
case, things can be more tricky if the overlay window is too large
(can happen if the screen is small and if you choose to position
candidate words horizontally). Someone (either the compositor or the
imserver) MUST know the both size of the window and the edge of the
screen in order to position the overlay correctly, and the im-server
MUST at least know the relative position to the cursor in order to set
the content of the overlay window correctly. So isn't it better to add
some interaction between compositor and im-server here?

>> I am also interested in whether this information can be passed and
>> used by another process (another wayland client)? It is possible now
>> to draw the input method interface by another process examples are
>> kimpanel[1] and kimtoy[2] both of which use a custom dbus protocol to
>> get information from the im server and draw the overlay window for it.
>> This is important for having a native user interface (kimpanel use
>> plasma-widget), keeping the im-server simple and reducing its
>> dependence (it doesn't need to worry about rendering or theming or any
>> kinds of eye candy etc.).
>>
>> Is it possible to add support for this (let a separate wayland client
>> to render the inteface)? Many im framework (including Fcitx and IBus)
>> already support this and it'll also be a lot easier for us to port our
>> imf to wayland (and keep X support at the same time) if this feature
>> can be supported.
>
> I have not really thought about input methods distributed over multiple
> processes/clients yet. But we will find a way to make it work.

For Fcitx, it is not that crucial since we have "classic ui" which is
in process (although kimpanel looks a LOOOOT better IMO). But IBus
will definitely require this feature. It is multi-process itself and
it gets input (from xim for example) and draw the interface in
different processes which communicate with each other through a
private dbus server... It is very likely to be a disaster for ibus if
this information cannot pass across processes/wayland client.

Also, what about the support for toolkit specific im-module? Can the
im-server get any information about the input surface and how to draw
the overlay correctly if it communicate with the (im) client through a
private protocol. (which is the case for most im server right now
afaik.)

THX.

>
> Jan Arne Petersen
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel

[1] http://wstaw.org/m/2012/08/31/plasma-desktopS22598.png
[2] http://wstaw.org/m/2012/08/31/plasma-desktopU22598.png