[Pixman] [PATCHv2 1/3] 'pixman_transform_bounds' fixed to match the rest of transform code

[Siarhei Siamashka]

From: Siarhei Siamashka <siarhei.siamashka at nokia.com>

This fixes the discrepancy between how FAST_PATH_SAMPLES_COVER_CLIP
flag is calculated and the code in fast_composite_scaled_nearest_*
functions.

Function 'pixman_transform_bounds' which is used for deciding whether
to set FAST_PATH_SAMPLES_COVER_CLIP flag, was doing 'pixman_transform_point'
call. While 'fast_composite_scaled_nearest_*' functions do initial addition
of 'pixman_fixed_1 / 2' constant, then call 'pixman_transform_point_3d' and
finally subtract 'pixman_fixed_e'.

As a result, two problems were possible:
1. Invalid memory accesses when fast_composite_scaled_nearest_* functions try
to fetch pixels outside the source image.
2. Missing FAST_PATH_SAMPLES_COVER_CLIP flag for the cases, when there is
actually no need to access pixels outside the source image. Causing fast
path functions not to be used and impacting performance.

As an additional note, function 'pixman_transform_point_3d' is truncating low
order bits in intermediate calculations losing some precision. But this
can't affect non-rotated nearest neighbor scaling.

The code was changed to transform bounds precisely. So that now any pixels
from the destination bounding box fit the calculated bounding box precisely
when translated to the source image. The older code was also translating
[x2,y2] point, which is unnecessary because it is actually outside the
bounding box.
---

The previous version of patch was not enough. It fixed the invalid memory
accesses problem. But still used to calculate transform bounds too
pessimistically, including more pixels than necessary and preventing
the use of fast path scaling functions in some cases. This problem is
now solved in v2.

 pixman/pixman-matrix.c |   60 ++++++++++++++++++++++++++---------------------
 1 files changed, 33 insertions(+), 27 deletions(-)

diff --git a/pixman/pixman-matrix.c b/pixman/pixman-matrix.c
index abdfa05..13def30 100644
--- a/pixman/pixman-matrix.c
+++ b/pixman/pixman-matrix.c
@@ -285,50 +285,56 @@ pixman_transform_bounds (const struct pixman_transform *matrix,
 {
     struct pixman_vector v[4];
     int i;
-    int x1, y1, x2, y2;
+    pixman_fixed_t x1 = F (b->x1), y1 = F (b->y1);
+    pixman_fixed_t x2 = F (b->x2), y2 = F (b->y2);
+
+    if (x2 > x1)
+	x2 -= pixman_fixed_1;
+    if (y2 > y1)
+	y2 -= pixman_fixed_1;
 
-    v[0].vector[0] = F (b->x1);
-    v[0].vector[1] = F (b->y1);
-    v[0].vector[2] = F (1);
+    v[0].vector[0] = x1 + pixman_fixed_1 / 2;
+    v[0].vector[1] = y1 + pixman_fixed_1 / 2;
+    v[0].vector[2] = pixman_fixed_1;
 
-    v[1].vector[0] = F (b->x2);
-    v[1].vector[1] = F (b->y1);
-    v[1].vector[2] = F (1);
+    v[1].vector[0] = x2 + pixman_fixed_1 / 2;
+    v[1].vector[1] = y1 + pixman_fixed_1 / 2;
+    v[1].vector[2] = pixman_fixed_1;
 
-    v[2].vector[0] = F (b->x2);
-    v[2].vector[1] = F (b->y2);
-    v[2].vector[2] = F (1);
+    v[2].vector[0] = x2 + pixman_fixed_1 / 2;
+    v[2].vector[1] = y2 + pixman_fixed_1 / 2;
+    v[2].vector[2] = pixman_fixed_1;
 
-    v[3].vector[0] = F (b->x1);
-    v[3].vector[1] = F (b->y2);
-    v[3].vector[2] = F (1);
+    v[3].vector[0] = x1 + pixman_fixed_1 / 2;
+    v[3].vector[1] = y2 + pixman_fixed_1 / 2;
+    v[3].vector[2] = pixman_fixed_1;
 
     for (i = 0; i < 4; i++)
     {
-	if (!pixman_transform_point (matrix, &v[i]))
+	if (!pixman_transform_point_3d (matrix, &v[i]))
 	    return FALSE;
 
-	x1 = pixman_fixed_to_int (v[i].vector[0]);
-	y1 = pixman_fixed_to_int (v[i].vector[1]);
-	x2 = pixman_fixed_to_int (pixman_fixed_ceil (v[i].vector[0]));
-	y2 = pixman_fixed_to_int (pixman_fixed_ceil (v[i].vector[1]));
-
 	if (i == 0)
 	{
-	    b->x1 = x1;
-	    b->y1 = y1;
-	    b->x2 = x2;
-	    b->y2 = y2;
+	    x1 = v[i].vector[0];
+	    x2 = v[i].vector[0];
+	    y1 = v[i].vector[1];
+	    y2 = v[i].vector[1];
 	}
 	else
 	{
-	    if (x1 < b->x1) b->x1 = x1;
-	    if (y1 < b->y1) b->y1 = y1;
-	    if (x2 > b->x2) b->x2 = x2;
-	    if (y2 > b->y2) b->y2 = y2;
+	    if (x1 > v[i].vector[0]) x1 = v[i].vector[0];
+	    if (x2 < v[i].vector[0]) x2 = v[i].vector[0];
+	    if (y1 > v[i].vector[1]) y1 = v[i].vector[1];
+	    if (y2 < v[i].vector[1]) y2 = v[i].vector[1];
 	}
     }
 
+    b->x1 = pixman_fixed_to_int (x1 - pixman_fixed_e);
+    b->y1 = pixman_fixed_to_int (y1 - pixman_fixed_e);
+    b->x2 = pixman_fixed_to_int (x2 - pixman_fixed_e) + 1;
+    b->y2 = pixman_fixed_to_int (y2 - pixman_fixed_e) + 1;
+
     return TRUE;
 }
 
-- 
1.6.4.4