[PATCH v8 xserver 6/7] xf86Cursor: Deal with rotation on GPU screens using a hw-cursor

[Hans de Goede]

When a slave-output is rotated the transformation is done on the blit
from master to slave GPU, so crtc->transform_in_use is not set, but we
still need to adjust the mouse position for things to work.

This commit modifies xf86_crtc_transform_cursor_position to not rely
on crtc->f_framebuffer_to_crtc, so that it can be used with GPU screens
to and always calls it for cursors on GPU screens.

Note not using crtc->f_framebuffer_to_crtc means that crtc->transform
will not be taken into account, that is ok, because when we've a transform
active hw-cursors are not used and xf86_crtc_transform_cursor_position
will never get called.

Signed-off-by: Hans de Goede <hdegoede at redhat.com>
---
Changes in v8:
-Fix reflection on rotated outputs (use xf86_crtc_rotate_coord_back again)
-Only call xf86_crtc_transform_cursor_position when rotating or reflecting
 (instead of always calling it for GPU screens)

Changes in v7:
-Do not use xf86_crtc_rotate_coord_back, it is not suitable for our purposes
-Modify xf86_crtc_transform_cursor_position instead of adding a new
 xf86_crtc_transform_gpu_cursor_position function

Changes in v5:
-New patch in v5 of this patch-series
---
 hw/xfree86/modes/xf86Cursors.c | 28 +++++++++++++++++-----------
 1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/hw/xfree86/modes/xf86Cursors.c b/hw/xfree86/modes/xf86Cursors.c
index c836534..3e9582b 100644
--- a/hw/xfree86/modes/xf86Cursors.c
+++ b/hw/xfree86/modes/xf86Cursors.c
@@ -367,16 +367,22 @@ xf86_crtc_transform_cursor_position(xf86CrtcPtr crtc, int *x, int *y)
     xf86CursorScreenPtr ScreenPriv =
         (xf86CursorScreenPtr) dixLookupPrivate(&screen->devPrivates,
                                                xf86CursorScreenKey);
-    struct pict_f_vector v;
-    int dx, dy;
-
-    v.v[0] = (*x + ScreenPriv->HotX) + 0.5;
-    v.v[1] = (*y + ScreenPriv->HotY) + 0.5;
-    v.v[2] = 1;
-    pixman_f_transform_point(&crtc->f_framebuffer_to_crtc, &v);
-    /* cursor will have 0.5 added to it already so floor is sufficent */
-    *x = floor(v.v[0]);
-    *y = floor(v.v[1]);
+    int dx, dy, width, height, rotation;
+
+    rotation = crtc->rotation & 0xf;
+    if (rotation == RR_Rotate_90 || rotation == RR_Rotate_270) {
+        width = crtc->mode.VDisplay;
+        height = crtc->mode.HDisplay;
+    } else {
+        width = crtc->mode.HDisplay;
+        height = crtc->mode.VDisplay;
+    }
+
+    *x = *x - crtc->x + ScreenPriv->HotX;
+    *y = *y - crtc->y + ScreenPriv->HotY;
+
+    xf86_crtc_rotate_coord_back(crtc->rotation, width, height, *x, *y, x, y);
+
     /*
      * Transform position of cursor upper left corner
      */
@@ -399,7 +405,7 @@ xf86_crtc_set_cursor_position(xf86CrtcPtr crtc, int x, int y)
     /*
      * Transform position of cursor on screen
      */
-    if (crtc->transform_in_use)
+    if (crtc->rotation != RR_Rotate_0)
         xf86_crtc_transform_cursor_position(crtc, &crtc_x, &crtc_y);
     else {
         crtc_x -= crtc->x;
-- 
2.9.3