[Piglit] [PATCH 2/2] framework/results: Fix subtests masking crashes
imirkin at alum.mit.edu
Tue May 10 13:41:33 UTC 2016
Shouldn't the test's result just be included into the "max" no matter what?
On Mon, May 9, 2016 at 5:46 PM, Dylan Baker <dylan at pnwbakers.com> wrote:
> In piglit when a test generates subtests we treat the "worst" subtest as
> the test status, and in most cases this works as expected. There is at
> least one case where this is not correct, and that is the case of crash.
> There are a couple of reasons that crash should not be masked. One is
> that it is generated by the framework when the test binary hits an
> assert or segfaults, or any number of similar cases. The second is that
> it may mean that all of the subtests did not run, as such we don't want
> the status to be masked by the "worst" subtest which would be fail at
> Signed-off-by: Dylan Baker <dylanx.c.baker at intel.com>
> framework/results.py | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
> diff --git a/framework/results.py b/framework/results.py
> index 469abeb..2095d90 100644
> --- a/framework/results.py
> +++ b/framework/results.py
> @@ -173,10 +173,13 @@ class TestResult(object):
> """Return the result of the test.
> If there are subtests return the "worst" value of those subtests. If
> - there are not return the stored value of the test.
> + there are not return the stored value of the test. There is an
> + exception to this rule, and that's if the status is crash; since this
> + status is set by the framework, and can be generated even when some or
> + all unit tests pass.
> - if self.subtests:
> + if self.subtests and self.__result != status.CRASH:
> return max(six.itervalues(self.subtests))
> return self.__result
> Piglit mailing list
> Piglit at lists.freedesktop.org
More information about the Piglit