[Pixman] [PATCH 0/5] Fix signed overflows

Søren Sandmann sandmann at cs.au.dk
Thu Dec 22 13:35:41 PST 2011


The following patches fix various cases of signed integer overflow,
which is undefined in C. The first patch was a result of

    https://bugs.freedesktop.org/show_bug.cgi?id=43906

in which Siarhei pointed out that clang miscompiles pixman due to a
signed overflow in some of the PDF operators. He also pointed to this
excellent tool:

     http://embed.cs.utah.edu/ioc/
       
which I used to find most of the other overflows.

Note that the trapezoid patch causes pixman to start rejecting
trapezoids where the top line is above one of the y1 coordinates of
the edges, or the bottom line is below one of the y2 coordinates. Such
trapezoids can be arbitrarily wide even with coordinates that
otherwise look reasonable.


Soren




More information about the Pixman mailing list