[PATCH xserver] xwayland: Fix backwards need_rotate logic (v2)

[Olivier Fourdan]

Hi,

On 20 February 2018 at 18:41, Jason Ekstrand <jason at jlekstrand.net> wrote:

> When xdg_output support was added to Xwayland, need_rotate parameter was
> added to output_get_new_size where true gave you the old pre-xdg_output
> behavior and false gave the new behavior.  Unfortunately, the two places
> where this is called, need_rotate was set backwards.  This caused input
> get clampped to the wrong dimensions.  Also, the logic for deciding
> whether or not to flip was wrong because, if need_rotate was false, it
> would always flip which is not what you want.
>
> v2 (Daniel Stone):
>  - Fix output_get_new_size so that it doesn't flip the dimensions when
>    need_rotate is false.
>
> Signed-off-by: Jason Ekstrand <jason at jlekstrand.net>
> Reviewed-by: Daniel Stone <daniels at collabora.com>
> Cc: Olivier Fourdan <ofourdan at redhat.com>
> ---
>  hw/xwayland/xwayland-output.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/hw/xwayland/xwayland-output.c b/hw/xwayland/xwayland-output.c
> index c593896..48faeb1 100644
> --- a/hw/xwayland/xwayland-output.c
> +++ b/hw/xwayland/xwayland-output.c
> @@ -126,7 +126,7 @@ output_get_new_size(struct xwl_output *xwl_output,
>  {
>      int output_width, output_height;
>
> -    if (need_rotate && (xwl_output->rotation & (RR_Rotate_0 |
> RR_Rotate_180))) {
> +    if (!need_rotate || (xwl_output->rotation & (RR_Rotate_0 |
> RR_Rotate_180))) {
>          output_width = xwl_output->width;
>          output_height = xwl_output->height;
>      } else {
> @@ -220,7 +220,7 @@ apply_output_change(struct xwl_output *xwl_output)
>      xwl_output->xdg_output_done = FALSE;
>
>      /* xdg-output sends output size in compositor space. so already
> rotated */
> -    need_rotate = (xwl_output->xdg_output != NULL);
> +    need_rotate = (xwl_output->xdg_output == NULL);
>
>      randr_mode = xwayland_cvt(xwl_output->width, xwl_output->height,
>                                xwl_output->refresh / 1000.0, 0, 0);
> @@ -390,7 +390,7 @@ xwl_output_remove(struct xwl_output *xwl_output)
>      struct xwl_output *it;
>      struct xwl_screen *xwl_screen = xwl_output->xwl_screen;
>      int width = 0, height = 0;
> -    Bool need_rotate = (xwl_output->xdg_output != NULL);
> +    Bool need_rotate = (xwl_output->xdg_output == NULL);
>
>      RRCrtcDestroy(xwl_output->randr_crtc);
>      RROutputDestroy(xwl_output->randr_output);
> --
> 2.5.0.400.gff86faf
>
> _______________________________________________
> xorg-devel at lists.x.org: X.Org development
> Archives: http://lists.x.org/archives/xorg-devel
> Info: https://lists.x.org/mailman/listinfo/xorg-devel
>

Hehe, yes, I was typing my reply pointing toward output_get_new_size() as
well but you were faster!

So v2 is:

Reviewed-by: Olivier Fourdan <ofourdan at redhat.com>

Thanks!
Olivier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.x.org/archives/xorg-devel/attachments/20180220/476293e7/attachment.html>