[Piglit] [V2 PATCH 04/12] summary.py: Split functionality out of NewSummary constructor

[Michel Dänzer]

On Fre, 2013-07-12 at 13:33 -0700, Dylan Baker wrote:
> That was intentional.
> 
> 
> My logic was this:
> a "Not Run" can mean two different things: 1) the test didn't exist
> during one run, 2) one set of results is much more comprehensive than
> another (say quick.tests vs quick.tests -t <only run a few tests>)
> 
> 
> In the first case I can accept the argument that a new test is in fact
> a change.
> In the latter case though the results would have 10000+ changes and
> regressions or fixes (depending on which order they were passed to
> summary-html). In this case these "Not Run" are actually more of "not
> known", and calling them a change is dangerous, as they might in fact
> not be changes.
> 
> 
> I decided that the latter case outweighed the former.

Not sure I can agree with that decision, but might it be possible to
deal with both cases by making the two meanings of "Not Run"
unambiguous?

Either way though, in the 'no changes' case, it would be useful to at
least see the names of the results directories and their numbers.
Otherwise, it would be easy to miss e.g. typos on the command line when
generating the summary.


-- 
Earthling Michel Dänzer           |                   http://www.amd.com
Libre software enthusiast         |          Debian, X and DRI developer