[Piglit] faster piglit start-up

[Dylan Baker]

Quoting Brian Paul (2017-04-12 21:03:47)
> On 04/12/2017 02:32 PM, Eric Anholt wrote:
> > Brian Paul <brianp at vmware.com> writes:
> >
> >> When we run piglit the all.py script traverses the entire tests/ and
> >> generated/tests/ directories looking for *.shader_test, *.vert, *.frag,
> >> etc. shaders to run.  In the generated_tests/ directory, there's about
> >> 60,000 entries to scan (ls -R generated_tests|wc)
> >>
> >> This takes some time, especially so with MinGW in a Windows VM.  Between
> >> that and other setup overhead it takes about 8 minutes on my system
> >> before any tests are run.  Ugh.  Linux is much faster but there's still
> >> a start-up lag.
> >>
> >> I'd like to trim this down.  The prime area I'm looking at is skipping
> >> swaths of tests when we know the particular feature is not supported by
> >> the GL driver.
> >>
> >> In framework/driver_classifier.py we run glxinfo to get the GL renderer
> >> string, etc.  I'm looking at expanding this to get the driver's GL and
> >> GLSL versions, extension list, etc.
> >>
> >> Then, in all.py when we're scanning the tests directory and we find
> >> somthing like "spec/arb_gpu_shader_fp64" which may not be supported by
> >> the driver, we can skip that directory.  For drivers which don't support
> >> all the latest GL 4.x extensions (like VMware's) we can save a lot of
> >> time by skipping subdirectory traversals and building the test list.
> >>
> >> I'm digging into this now but it'll probably be a few days before I'll
> >> have something to review.
> >>
> >> I'd be interested to hear any other ideas to reduce start-up time.
> >
> > vc4 is also terrible to test these days due to the explosion of
> > generated shader_tests we have -- I've got them hacked out of my builds
> > due to FS space used and the minutes it takes to start piglit when
> > they're present.
> >
> > For a solution to the startup time, I wish we would just build our whole
> > test list using filesystem walks or whatever at piglit build time,
> > instead of executing python to build the test list per piglit run.

I have such a branch! I'll work on it today and see what I can do to get it out
for review.

> 
> I like that idea.
> 
> Though, I'd still like a way to skip running all the tests for 
> "spec at GL_EXT_foobar" if we know that GL_EXT_foobar isn't supported by 
> the driver we're testing.  Or similarly for GL x.y or GLSL x.y.
> 
> If we have N GL_EXT_foobar tests, it's kind of crazy to start N 
> processes, setup a GL context then return SKIP as soon as 
> piglit_require_extension("GL_EXT_foobar") fails.  N is pretty large for 
> some extensions nowadays.

Piglit already has fast skipping that does just that for shader and glsl parser
tests, it's about 1000x faster than starting the process (0.1 seconds to 0.0001
seconds). It's possible to extend fast skipping to work for c tests, but we'd
need metadata to tell piglit what extensions or gl/glsl versions are required.

> 
> I know we can pass a bunch of -x options to piglit-run.py to effectively 
> do that, but that's a lot of busywork.
> 
> -Brian
> 
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/piglit


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: signature
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20170413/bfc1051c/attachment.sig>