Atomic KMS API lacks the ability to set cursor hot-spot coordinates

[Hans de Goede]

Hi,

On 3/19/20 1:58 PM, Pekka Paalanen wrote:
> On Thu, 19 Mar 2020 12:49:27 +0100
> Hans de Goede <hdegoede at redhat.com> wrote:
> 
>> Hi,
>>
>> On 3/19/20 11:00 AM, Pekka Paalanen wrote:
>>> On Wed, 18 Mar 2020 15:28:02 +0100
>>> Hans de Goede <hdegoede at redhat.com> wrote:
>>>    
>>>> ATM the Atomic KMS API lacks the ability to set cursor hot-spot
>>>> coordinates. Mutter (and Weston) have tried to emulate this by shifting
>>>> the coordinates for where to draw the cursor by the hotspot-coordinates
>>>> and always using 0x0 for the hotspot.
>>>>
>>>> But this breaks the so called "seamless mouse mode" for virtual-machines
>>>> and there really is no way to fix this but to allow passing the proper
>>>> hotspot coordinates to the virtual gfx-card.
>>>>
>>>> Seamless-mode consists of 2 parts:
>>>>
>>>> 1) Letting the VM-viewer window-system draw the cursor as it normally
>>>> would draw it.
>>>>
>>>> 2) Giving absolute coordinates of the mouse to the VM by e.g. emulating
>>>> an USB drawing tablet. These coordinates come from translating the
>>>> coordinates where the VM-viewer window-system is drawing the cursor
>>>> to an absolute position using the top left corner of the view as 0x0
>>>> and the bottom right corner as max-abs-x,max-abs-y.
>>>
>>> Hi,
>>>
>>> is the VM-viewer then hiding or autonomously moving what the display
>>> server inside VM has put on the KMS cursor plane?
>>
>> Yes and no, it is not the VM-viewer which is hiding what the
>> display-server inside the VM has put on the KMS cursor plane,
>> the VM-viewer negotiates seamless mouse mode with the hypervisor
>> and then the hypervisor just ignores the cursor-plane except for
>> sending "sprite" changes to the VM-viewer.
> 
> Hi,
> 
> I don't think I understand what you're saying, but I assume that I was
> right in that the VM cursor plane content will not be shown always
> exactly in the very position the compositor inside the VM puts it.

Right, when seamless mouse mode is enabled the VM's cursor plane will
not be shown *at all*, instead the VM-viewer's window-system's
cursor plane is shown.

Sprite changes (normal cursor, caret, window-resize cursor, etc.)
are forwarded from the VM-s cursor-plane to the VM-viewer
which then requests that as cursor to the window-system under
which the VM-viewer is running.

> Maybe the example further below explain the issue I envision.
> 
>>> If so, sounds like hilarity would ensue with Weston.
>>>
>>> Weston does not actually know what a cursor is. Weston will happily put
>>> any arbitrary non-cursor content onto the KMS cursor plane if it happens
>>> to fit. It's just that usually there is a small surface top-most that
>>> ends up on the cursor plane, and that surface accidentally happens to
>>> be a cursor by Wayland protocol.
>>>
>>> It's not difficult to get e.g. weston-simple-shm window to be shown on
>>> the KMS cursor plane: just hide the real cursor from the client.
>>>
>>> No, it's not an oversight. It is called "making maximal use of the
>>> available KMS resources" by using KMS planes as much as possible to
>>> avoid compositing by rendering with Pixman or OpenGL.
>>
>> Yes it sounds like this will break with Weston, note that it already
>> is broken in Weston, if you run e.g. Fedora 32 beta + its Weston
>> package inside a VirtualBox VM then start gnome-terminal (so
>> that you get a caret cursor instead of the default one) and try to
>> select text. This will result in the wrong text being displayed
>> because Weston does not relay cursor hotspot info to the GPU,
>> also see:
>> https://gitlab.gnome.org/GNOME/mutter/issues/1094
>>
>> Where the symptoms of this are described in more detail
>> (they are identical for Weston and mutter).
> 
> Right, that's the problem with the hotspot.

Ack.

>> Fixing this will require the discussed KMS atomic API changes
>> and also changes on the Weston and mutter side to pass through
>> the hotspot info.
> 
> The problem I am referring to is that to the user looking at the
> VM-viewer, suddenly an arbitrary application window (e.g.
> weston-simple-shm) starts to act as if it was the cursor, when there is
> no real cursor shown. You have a random window unexpectedly moving
> around, as if you had started dragging it around with your mouse.

Correct.
> The only way to fix that is to stop Weston from putting non-cursor
> content on the cursor plane.

Correct.

> It sounds like your VM-viewer makes the assumption that the pointer
> input device it delivers to the VM is the one that will control the KMS
> cursor plane position inside the VM. Is that right?

Correct.

> What if the desktop inside the VM is controlled by a remote, e.g. VNC?
> Then the input events to the VM are completely unrelated to the
> expected motion of the cursor. Do you just tell the users to stop using
> the seamless mode in that case?

A VNC viewer (which is not using seamless mode itself) has the same issue
of a mismatch between the cursor position of the window-system it is a client
of and the cursor position of the window system inside the VM.

A VNC viewer typically works around this by changing the window-system
cursor to transparent and drawing its own cursor, the transparent sprite
or disabling of the cursor-plane will get forwarded to the window-system
under which the VM-viewer runs so this will work fine.

A real problem though is the absolute input mode, when e.g. the VNC
viewer is not using something like seamless mouse mode it will want
to do the usual (nasty) tricks of confining the pointer to the window
and warping it to the center after each pointer move so that mouse
can be moved "endlessly' inside the window. This requires relative
input, so emulating a mouse instead of say a drawing tablet, which
breaks seamless mode.

So although the way the cursor is drawn typically does not require
disabling seamless mode, the input problems do require disabling
seamless mode.

In my experience (I have worked on VDI (spice) a couple of years),
running a VM-viewer inside a VM is something which not a lot of
people do. I guess the fact that this works quite poorly might
be one of the reasons people do not do this.

But all VMs I'm familiar with allow disabling seamless mode for
compatibility with corner cases like this. This does lead to a
seriously degraded user-experience though. For just using an
office suit or browser inside a VM seamless mode really is a
lot more pleasant for the user.  I still remember when seamless
mode first got introduced, in the beginning it used to not
always work and fixing the fallback to the confine + warp tricks
then always was a serious itch which I tried to scratch quickly.
Breaking seamless mode also is a good way to quickly accumulate
a lot of bug reports from end users in my experience.

> What if display servers stop using the cursor plane completely, because
> people may hit a case where a VM-viewer makes the wrong assumption about
> which input device is associated to which cursor plane inside the VM?

The confine + warp trick is typically the default mode and only
if the guest indicates through e.g. a guest-agent process that
seamless mode is supported then seamless mode is enabled.

IOW the VM is careful to not enable it when it might not work.

OTOH most distros now ship with the necessary agents installed
by default, so e.g. a F32 beta install will automatically use
seamless mode under QEMU/KVM (spice display protocol) or
VirtualBox.  If a user uses say F32 + a vnc-viewer inside it
which needs to do confine + warp tricks then the user will have
to disable seamless mode manually. In my prior experience
working on VDI this is usually something which users are
understanding of and not a problem.

Regards,

Hans