[PATCH v7 xserver 6/7] xf86Cursor: Deal with rotation on GPU screens using a hw-cursor

[Hans de Goede]

When a slave-output is rotated the transformation is done on the blit
from master to slave GPU, so crtc->transform_in_use is not set, but we
still need to adjust the mouse position for things to work.

This commit modifies xf86_crtc_transform_cursor_position to not rely
on crtc->f_framebuffer_to_crtc, so that it can be used with GPU screens
to and always calls it for cursors on GPU screens.

Note not using crtc->f_framebuffer_to_crtc means that crtc->transform
will not be taken into account, that is ok, because when we've a transform
active hw-cursors are not used and xf86_crtc_transform_cursor_position
will never get called.

Signed-off-by: Hans de Goede <hdegoede at redhat.com>
---
Changes in v7:
-Do not use xf86_crtc_rotate_coord_back, it is not suitable for our purposes
-Modify xf86_crtc_transform_cursor_position instead of adding a new
 xf86_crtc_transform_gpu_cursor_position function

Changes in v5:
-New patch in v5 of this patch-series
---
 hw/xfree86/modes/xf86Cursors.c | 42 +++++++++++++++++++++++++++++++-----------
 1 file changed, 31 insertions(+), 11 deletions(-)

diff --git a/hw/xfree86/modes/xf86Cursors.c b/hw/xfree86/modes/xf86Cursors.c
index 8437000..f638452 100644
--- a/hw/xfree86/modes/xf86Cursors.c
+++ b/hw/xfree86/modes/xf86Cursors.c
@@ -384,16 +384,35 @@ xf86_crtc_transform_cursor_position(xf86CrtcPtr crtc, int *x, int *y)
     xf86CursorScreenPtr ScreenPriv =
         (xf86CursorScreenPtr) dixLookupPrivate(&screen->devPrivates,
                                                xf86CursorScreenKey);
-    struct pict_f_vector v;
-    int dx, dy;
-
-    v.v[0] = (*x + ScreenPriv->HotX) + 0.5;
-    v.v[1] = (*y + ScreenPriv->HotY) + 0.5;
-    v.v[2] = 1;
-    pixman_f_transform_point(&crtc->f_framebuffer_to_crtc, &v);
-    /* cursor will have 0.5 added to it already so floor is sufficent */
-    *x = floor(v.v[0]);
-    *y = floor(v.v[1]);
+    int dx, dy, t;
+
+    *x = *x - crtc->x + ScreenPriv->HotX;
+    *y = *y - crtc->y + ScreenPriv->HotY;
+
+    if (crtc->rotation & RR_Reflect_X)
+        *x = crtc->mode.HDisplay - *x - 1;
+    if (crtc->rotation & RR_Reflect_Y)
+        *y = crtc->mode.VDisplay - *y - 1;
+
+    switch (crtc->rotation & 0xf) {
+    case RR_Rotate_0:
+        break;
+    case RR_Rotate_90:
+        t = *x;
+        *x = *y;
+        *y = crtc->mode.VDisplay - t - 1;
+        break;
+    case RR_Rotate_180:
+        *x = crtc->mode.HDisplay - *x - 1;
+        *y = crtc->mode.VDisplay - *y - 1;
+        break;
+    case RR_Rotate_270:
+        t = *x;
+        *x = crtc->mode.HDisplay - *y - 1;
+        *y = t;
+        break;
+    }
+
     /*
      * Transform position of cursor upper left corner
      */
@@ -408,6 +427,7 @@ static void
 xf86_crtc_set_cursor_position(xf86CrtcPtr crtc, int x, int y)
 {
     ScrnInfoPtr scrn = crtc->scrn;
+    ScreenPtr screen = scrn->pScreen;
     xf86CrtcConfigPtr xf86_config = XF86_CRTC_CONFIG_PTR(scrn);
     xf86CursorInfoPtr cursor_info = xf86_config->cursor_info;
     DisplayModePtr mode = &crtc->mode;
@@ -416,7 +436,7 @@ xf86_crtc_set_cursor_position(xf86CrtcPtr crtc, int x, int y)
     /*
      * Transform position of cursor on screen
      */
-    if (crtc->transform_in_use)
+    if (crtc->transform_in_use || screen->isGPU)
         xf86_crtc_transform_cursor_position(crtc, &crtc_x, &crtc_y);
     else {
         crtc_x -= crtc->x;
-- 
2.9.3