[Piglit] Seperate NOTRUN and SKIP from fixes and regressions

[Tom Stellard]

On Mon, Mar 17, 2014 at 12:06:09PM -0700, Dylan Baker wrote:
> This series separates the SKIP and NOTRUN status changes from fixes and
> regressions and puts them into separate categories, enabled and
> disabled.
> 
> This makes two workflows possible where previously only one was:
> 1) creating baselines that are full tests, and then using small focused
>    runs to compare specific tests
> 2) running full comparisons, and being able to see when new tests are
>    enabled or tests are disabled
> 
> 
> Patches 1-3 add test to demonstrate current behavior or to model future
> behavior
> 
> Patch 4 makes NOTRUN and SKIP a special subclass which when compared
> using numerics (not == or !=) will resolve False
> 
> Patch 5 Plumbs it all in

I did some basic testing of the patches and they seem to work OK.

This will help eliminate one of the sources of false positives
on the buildbot I'm experimenting with, which is when someone adds a
failing test to piglit, because 'NOTRUN' -> 'FAIL' is currently a
regression.  Thanks for working on this.

Tested-by: Tom Stellard <thomas.stellard at amd.com>


> 
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit