[Spice-devel] [PATCH xf86-qxl] Fix crash when calling CrtcRotate()

[Rob Clark]

On Fri, Oct 10, 2014 at 4:11 AM, David Airlie <airlied at redhat.com> wrote:
>
>
>
> ----- Original Message -----
>> From: "Rob Clark" <robdclark at gmail.com>
>> To: "Marc-André Lureau" <marcandre.lureau at gmail.com>
>> Cc: "Christophe Fergeau" <cfergeau at redhat.com>, "Marc-André Lureau" <marcandre.lureau at redhat.com>, "spice-devel"
>> <spice-devel at lists.freedesktop.org>, airlied at redhat.com
>> Sent: Friday, 10 October, 2014 5:51:28 PM
>> Subject: Re: [Spice-devel] [PATCH xf86-qxl] Fix crash when calling CrtcRotate()
>>
>> On Thu, Oct 9, 2014 at 12:29 PM, Marc-André Lureau
>> <marcandre.lureau at gmail.com> wrote:
>> > Hi
>> >
>> > Rob and Dave, I hope you can shed some light here.
>> >
>> > On Thu, Sep 18, 2014 at 1:26 PM, Christophe Fergeau <cfergeau at redhat.com>
>> > wrote:
>> >>
>> >> A bit confused that we try to call something that cannot work in our
>> >> code. Can drmmode_set_mode_major report an error rather than attempting
>> >> to rotate?
>> >
>> >
>> > It seems src/qxl_drmmode.c is based on xf86-video-modesetting code, also
>> > calling CrtcRotate:
>> > http://cgit.freedesktop.org/xorg/driver/xf86-video-modesetting/tree/src/drmmode_display.c
>> >
>> > Until very recently, it also implemented those stub functions:
>> > http://cgit.freedesktop.org/xorg/driver/xf86-video-modesetting/commit/src?id=91571cce03adc58a10b5d76307629efc7c994c3e
>> >
>> > I wonder if xf86-video-modesetting will reach the same crash now (since it
>> > also has no crtc->driverIsPerformingTransform).
>>
>> So, I guess I'm missing some context here, but in xf86CrtcScreenInit()
>> if those two hooks are not implemented, it should only expose support
>> for zero degree rotation (ie. rotation not supported)..  what crash
>> are you seeing?
>
> I'm not near the code at the moment, but I seem to remember the X server doing something stupid like calling xf86CrtcRotate,
> if the crtc won't fit in the root pixmap, so it assumes it must be rotated.

oh, whoops.. yeah, we probably shouldn't do that for -modesetting.
For -qxl, I'm not sure.. it depends on whether it actually does
support rotation (in which case it should either implement those hooks
or set driverIsPerformingTransform I think)

BR,
-R