[Piglit] [Bug 91680] Different stats in output reports

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=91680

Dylan <baker.dylan.c at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |NOTABUG

--- Comment #1 from Dylan <baker.dylan.c at gmail.com> ---
This isn't actually a bug. What's happening is that 'piglit run' and 'piglit
summary' are counting two different things.

Piglit run counts the number of tests as defined in the profile (in your case
tests/all.py), it counts them up and sees that there are 28030 Test derived
objects in the python file.

Piglit summary counts the number of tests + subtests as a total (tests with
subtests are treated as a group rather than as a test), so while there were
28030 Test.run() calls, there were actually 28904 test + subtest results
generated.

There is no way for the runner to know if a test has subtests or how many it
has until it has finished running that test; and I prefer to keep the number of
tests counted deterministic at start time (ie, not changing as the run goes,
which would be confusing when running a summary against two incomplete
results).

I've attached a very simple results file that hopefully will make it clearer
how this works. I'll also send a patch that updates the README file with this
information.

Sorry for the confusion.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20150818/a2cd4172/attachment.html>