[Piglit] Add platform based test skipping to python

[Emil Velikov]

Hi Dylan,

On 04/10/14 01:57, Dylan Baker wrote:
> We have recently started using jenkins here at Intel to do regression
> testing across multiple generations of hardware. Because of this we have
> become concerned with the large number of tests that fail and the large
> number of tests that skip, specifically on older platforms.
> 
> This series is the first 15 patches shaved off of a larger series I'm
> working on to add better test skipping in the framework.
> 
> I have targeted the framework as the best place to skip tests for a
> couple of reasons. First, it allows us to use metadata in the various
> profiles as skip criteria; in this series platforms (glx, egl, gbm, etc)
> are used to skip tests when that platform isn't available. In a later
> patches I'm working on we're able to use GL support as that criteria,
> allowing old platforms to skip GL features that were not part of that
> hardware's targeted GL API.
> 
> Doing this work I took some time to cleanup the piglit Test code, I
> created a test package to hold the growing number of Test classes, split
> some code into Mixins, extended the early return code in Test.run() to
> be more versatile, split the PiglitTest class into PiglitGLTest and
> PiglitCLTest classes, pulled the code that adds -fbo and -auto into
> PiglitGLTest's command getter, and cleaned up some potential bugs.
> 
> All of these changes result in better code, better code sharing between
> OpenGL and OpenCL tests (which means basically OpenCL doesn't run OpenGL
> specific code which only increases their runtime), and ultimately the
> ability to skip tests based on platform.
> 
> This series implements fast skipping for two types of tests, glx tests
> on non glx, and egl tests on glx. I was somewhat unsure about the right
> way to skip the egl tests. Currently all platforms except pure glx
> support egl (x11_egl, mixed_glx_egl, and wayland always do), or can be
> configured with egl support (gbm). I chose to exclude glx, but I would
> be equally opened to include x11_egl, mixed_glx_egl, gbm, and wayland
> instead.
> 
I would prefer if we go with the explicit "requires_platform", otherwise
as we add wgl and cgl we'll end up in a mess :\ Perhaps adding a meta
platform "egl" to encapsulate x11_egl, mixed_glx_egl, gbm, and wayland
might be a good idea ?

Thanks
Emil

> The difference in runtime for a test that fast skips vs one that slow
> skips is dramatic. I measured that it takes about .00005 seconds for a
> test to fast skip, and about .1 seconds to slow skip. That adds up very
> quickly if your platform skips 20000 tests.
> 
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit
>