[Piglit] [PATCH 1/1] results.py: Print warning when overwriting subtest results

[Dylan Baker]

On Wednesday, September 03, 2014 09:44:08 PM Ilia Mirkin wrote:
[snip]
> Hmmmm.... right. In the Test constructor:
> 
>         self.result = TestResult({'result': 'fail'})
> 
> And in case that wasn't there, right before the
> self.interpret_result() call in Test.run():
> 
>         self.result['result'] = 'fail'

I could have sworn that I deleted that. I'll send a patch to do that

> 
> (which is actually a little wrong because it needs to be one of those
> status things, but nothing cares at exec time). It does seem like
> that's unnecessary... Dylan? Seems like that could just be replaced
> with
> 
> self.interpret_result()
> if 'result' not in self.result:
>   self.result['result'] = status.FAIL

Maybe we should overwrite TestResult.__setitem__ to set result to a
status class

> 
> >
> >>
> >> Printing stuff right there goes against the logging logic... I think
> >> TestResults are also used during test execution, not just at summary
> >> time... Dylan may have ideas on how to expose the issue properly.
> >
> > These need to be detected at execution time otherwise only the last
> > result is stored in the results file and we loose information (and hide
> > possible failures).
> > if there is a better way to report these, I don;t mind changing it.

This feels like working around bugs in our own software. My opinion on
the matter is that sub-tests should not have the same name, and if they
do that's a bug. I think an assert should cover this, if there is a
duplicate, it hits the assert and the test fails with an assertion.

I'm also rather against printing a warning. If being able to do this
more than once, why don't we add a new method that tests for duplicate
sub-tests, and returns a report. Do something like count the number of
times a specific test name is reported?

[snip]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20140904/385c8359/attachment.sig>