[Intel-gfx] [PATCH i-g-t v3 2/4] tests/ chamelium: Remove the frame dump tests

[Martin Peres]

On 05/07/17 23:53, Lyude Paul wrote:
> NAK. You're right that these don't actually give us any advantage over
> just using CRCs and are just slower, however I left these tests in here
> moreso just so we had something to actually test the frame dumping
> functions so that we could avoid regressing them by accident since
> we're the only users of those functions right now.
> 
> If I recall properly, isn't there a list of tests in igt's source that
> they use for determining which tests to run on the CI? I think a better
> solution would be to just disable this for CI runs, and maybe add some
> comments pointing out that this test is only really useful for
> developers making changes to the chamelium library.

The general direction of IGT is to make sure that all tests are being 
executed by default, very much alike piglit.

I however do understand what you mean and I would then propose that we 
start defining naming conventions that allow filtering tests based on 
how level directive. For instance, we could have every test that need it 
export a slow and fast version, so as to easily be able to filter stuff 
out. I guess we need to start a separate thread for that.

As for this particular test, I would say that it could be simplified to 
prevent downloading so many images (all the exported resolutions * all 
the connected connectors). We could instead test one resolution per 
connector and call it good-enough. By better testing stuff, we just 
increase the likeliness of the test being completely blacklisted due to 
its excessive execution time. In this case, the better is the enemy of good.