[pixman] [Pixman] [PATCH 1/2] Extend blitters test to first check known failure cases

[Siarhei Siamashka]

On Thu, Apr 8, 2010 at 2:39 AM, Søren Sandmann <sandmann at daimi.au.dk> wrote:
> From: Søren Sandmann Pedersen <ssp at redhat.com>
>
> This serves to make sure we don't accidentally reintroduce old bugs.

The problem (or a feature) of the current tests is that crc32 can
change any time and such lists of known old failure cases of may be
invalidated any time in the future. A proper solution would be to
generate reproducible traces for the failed testcases (record all the
sequence of function calls, their parameters and input data, plus
store the expected output data).

I have a work-in-progress branch here intended to improve pixman
regression testing (and messes up all the checksums):
http://cgit.freedesktop.org/~siamashka/pixman/log/?h=tests-refactoring

The current problems if the regression tests include:
1. missing a8b8g8r8 format for mask in blitters-test
2. scaling-test coverage is actually not good enough (it's
embarassing, but it did not take into account that the random number
generator has a limited range for generated values).

Also this branch tries to utilize multiple CPU cores via OpenMP to
speed up the tests. This seems to be a very simple change, but works
fine and is supposed to be portable.

I was distracted from this branch when the the regressions on older
ARM cores had to be addressed just before pixman 0.18.0 release. But
now it seems to be a good time to finally finish this stuff.

>  test/blitters-test.c |   43 +++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 43 insertions(+), 0 deletions(-)
>
> diff --git a/test/blitters-test.c b/test/blitters-test.c
> index 5e33031..25b3c7b 100644
> --- a/test/blitters-test.c
> +++ b/test/blitters-test.c
> @@ -437,6 +437,46 @@ initialize_palette (void)
>        palette.ent[i] = lcg_rand() & 0xff;
>  }
>
> +typedef struct
> +{
> +    uint32_t nth;
> +    uint32_t crc;
> +} testcase_t;
> +
> +static testcase_t known_failures[] =
> +{
> +    { 592654, 0x06BF6ADE },
> +};
> +
> +static int
> +run_known_failures (void)
> +{
> +    int i;
> +
> +    for (i = 0; i < sizeof (known_failures) / sizeof (known_failures[0]); ++i)
> +    {
> +       testcase_t *testcase = &(known_failures[i]);
> +       uint32_t crc;
> +
> +       crc = test_composite (0, testcase->nth, 0);
> +
> +       if (crc != testcase->crc)
> +       {
> +           crc = test_composite (0, testcase->nth, 1);
> +
> +           printf ("Test case %d failed\n"
> +                   "CRC:      %08x\n"
> +                   "Expected: %08x\n",
> +                   testcase->nth,
> +                   crc,
> +                   testcase->crc);
> +           return 1;
> +       }
> +    }
> +
> +    return 0;
> +}
> +
>  int
>  main (int argc, char *argv[])
>  {
> @@ -446,6 +486,9 @@ main (int argc, char *argv[])
>
>     initialize_palette();
>
> +    if (run_known_failures ())
> +       return 1;
> +
>     if (argc >= 3)
>     {
>        n1 = atoi (argv[1]);
> --
> 1.7.0.1