XRandR transform (keystone correction) not working as I would expect

[Glynn Clements]

AJ Heller wrote:

> I implemented a transformation matrix to pinch the top of my screen 100px
> on each side. My hope is for the resulting image to be an isosceles
> trapezoid (trapezium) with the smaller base on top, larger base on bottom
> filling the width of the screen, and with no part of the image missing or
> drawn off the screen.
> 
> After transforming, the top remains fixed at the width of the screen, and
> the bottom of the trapezoidal image is drawn off the screen on either side.
> I could be confused about what the transform should be doing, but otherwise
> I think this is a bug.
> 
> Have I setup my transformation incorrectly? Should I file a bug report?

> I'm working with 1440x900 resolution. I mapped the four corners to my
> desired output coordinates:
> 
> (0,0) => (0,0)
> (1440,0) => (1440,0)
> (0,900) => (100,900)
> (1440,900) => (1340,900)

> My transformation matrix is:
> 1, 0.30769, 0
> 0, 1.38462, 0
> 0, 0.00043, 1

Based upon the xrandr manual page and your pictures, that's not what
you want, and will produce the results you describe.

First, the transformation converts from output coordinates to input
coordinates. So the above will result in the bottom corners of the
screen being mapped 100 pixels in from the edge (i.e. you'll lose 100
pixels at each side). Except, it's actually 200 pixels for the matrix
which you give.

Second, 0,0 is the top-left corner, not the bottom-left (this isn't
stated in the manual page, but it explains why the the oversized edge
is the bottom edge rather than the top edge).

If I'm understanding you (and xrandr) correctly, the matrix you want
is:

1.16129, 0.12903, -116.12903
0.     , 1.16129,    0.     
0.     , 0.00018,    1.     

This will map:

 output        input

 100,   0       0,   0
1340,   0    1440,   0
   0, 900       0, 900
1440, 900    1440, 900

-- 
Glynn Clements <glynn at gclements.plus.com>