[Intel-gfx] The whole round of i-g-t testing cost too long running time

[Yang, Guang A]

> I think stopping the tests after 10 minutes is ok, but in general the point of
> stress tests is to beat on the kernel for corner cases. E.g.
> even with todays extensive set of stress tests some spurious OOM bugs can
> only be reproduced in 1 out of 5 runs. Reducing the test time could severely
> impact the testing power of a test, so I'm vary for doing that.

[Guang YANG] Agree with you, but whether stop these stress testing after 10M if it not finished will impact the effort?

> But there are tricks to speed up some tests which shouldn't affect the power of
> the testcase to find bugs, and we should definitely look into those.
> 
> Does this mean that due to PRTS we now have fewer machines running tests
> on drm-intel-nightly? I've thought the idea is to share machines on an
> as-needed basis, with -nightly testing getting priority?

[Guang YANG] No, we haven’t changed the scale for existed nightly testing resources ,and the PRTS can supplement our nightly well with more old platforms(ILK,PNV), but we still lack of BDW/BYT resources in both PRTS and nightly. 
			These new platforms are important and most of the bug reported on them. Running tests, checking patches, doing the bisect work keep the machine resources always busy. Shuang still hard work on sharing machines between PRTS and nightly. 
> 
> 
> Yeah keeping your overall test-runner infrastructure makes sense. The idea
> behind my proposal to use piglit to execute the individual tests is to share
> analysis scripts. That won't make the tests run any faster, but it should (in the
> long term at least) speed up the triaging a lot. And the high amount of time
> required for bug triaging also seems to be an issue for you guys.

[Guang YANG] Great, improve the analysis script or tool will be comfortable for QA, We can catch the regression more accurately for such as BackTrace. Do you have any plans? QA can also take over some tasks. 

> I agree that for a quick sanity test a reduced test set makes sense.
> Which is why we have a testcase naming convention which can be used
> together with the piglit -x and -t flags. I do that a lot when developing things.
> 
> But for regression testing imo only the full test suite makes sense, otherwise
> we just have a false sense of security. I.e. if the full set means we can only run
> it every 2 days then I prefer that over running only a subset. Also very often
> there are other issues delaying the time between when a buggy patch was
> committed and when the full bug report is available, so imo the 10h runtime
> isn't too bad from my pov really.

> -Daniel
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> +41 (0) 79 365 57 48 - http://blog.ffwll.ch[Guang YANG]