[Spice-devel] [PATCH xf86-qxl] Fix crash when calling CrtcRotate()

[Marc-André Lureau]

Although the driver doesn't support rotation, drmmode_set_mode_major may
call CrtcRotate when the crtc mode doesn't fit on screen.

This will trigger a call to crtc->shadow_allocate() that will result in
SEGV since our driver doesn't implement it. Instead, let's fail
gracefully.

PS: I didn't manage to figure out why the mode didn't fit on screen when
vncviewer try to resize the display. When debugging Xorg with gdb, it
either crashes gdb when inspecting *crtc->scrn in
drmmode_set_mode_major, or make the vnc connection "freeze": the
problematic resize code isn't reached when breaking on
drmmode_set_mode_major() and drmmode_xf86_crtc_resize().

http://bugzilla.redhat.com/1067709
---
 src/qxl_drmmode.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/src/qxl_drmmode.c b/src/qxl_drmmode.c
index 42347e6..4d04404 100644
--- a/src/qxl_drmmode.c
+++ b/src/qxl_drmmode.c
@@ -331,6 +331,19 @@ drmmode_crtc_gamma_set(xf86CrtcPtr crtc, uint16_t *red, uint16_t *green,
 	drmModeCrtcSetGamma(drmmode->fd, drmmode_crtc->mode_crtc->crtc_id,
 			    size, red, green, blue);
 }
+
+static void *
+drmmode_shadow_allocate(xf86CrtcPtr ctrc, int width, int height)
+{
+  return NULL;
+}
+
+static PixmapPtr
+drmmode_shadow_create(xf86CrtcPtr ctrc, void *data, int width, int height)
+{
+  return NULL;
+}
+
 static const xf86CrtcFuncsRec drmmode_crtc_funcs = {
     .dpms = drmmode_crtc_dpms,
     .set_mode_major = drmmode_set_mode_major,
@@ -341,6 +354,9 @@ static const xf86CrtcFuncsRec drmmode_crtc_funcs = {
     .load_cursor_argb = drmmode_load_cursor_argb,
 
     .gamma_set = drmmode_crtc_gamma_set,
+    .shadow_allocate = drmmode_shadow_allocate,
+    .shadow_create = drmmode_shadow_create,
+
     .destroy = NULL, /* XXX */
 };
 
-- 
1.9.3