[Piglit] [PATCH 2/2] framework/results: Fix subtests masking crashes

[Dylan Baker]

Quoting Ilia Mirkin (2016-05-10 14:32:59)
> On Tue, May 10, 2016 at 5:32 PM, Dylan Baker <dylan at pnwbakers.com> wrote:
> > Quoting Ilia Mirkin (2016-05-10 06:41:33)
> >> Shouldn't the test's result just be included into the "max" no matter what?
> >>
> >
> > Well, the problem is that a lot of tests were retrofitted with subtests,
> > and now the overall status is bunk, and for new tests we generally don't
> > add an overall result when there's subtests.
> 
> It's surprising if the status of a test says fail but piglit-summary
> reports it as pass. In the case where one of the subtests fails but
> the overall status is pass, this would still say fail, no?
> 
> What situation are you protecting against by only incorporating the
> overall status on crash?

I have this gut feeling that says skip or notrun might play havok here.
But I guess I should look test it and see if that's the case.

The other thing I guess is that there's only about 100 tests (or, c/c++
files, really) using subtests. So maybe it would just be easier to audit
them to make sure that they do the right thing and just add the overall
result to the max call.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: signature
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20160510/ca29d445/attachment-0001.sig>