[Piglit] [PATCH v3 1/2] summary.py: Treat subtests as groups

[Kenneth Graunke]

On 11/14/2013 10:26 AM, Dylan Baker wrote:
> This patch causes tests with subtests to be treated as a group, rather
> than as a test. This means that the status the test itself stores will
> be overwritten by those in the subtest.
> 
> There is one oddity about this to be aware of; a test with subtests that
> crashes or fails before any of the subtests run will report a fraction
> of 0/1 with the appropriate color, even though all of the subtests will
> report Not Run.
> 
> v2: - Add subtests to the results file as full tests (the internal view
>       of the json), without this they will not appear in changes, fixes,
>       etc
>     - Render the background color of Not Run tests correctly in HTML
>     - Apply subtest fractions down the stack
> v3: - Don't generate a link for Not Run tests in html page
> 
> Tested-by: Tom Stellard <thomas.stellard at amd.com>
> Signed-off-by: Dylan Baker <baker.dylan.c at gmail.com>

This patch breaks summary generation for me - on the most basic of cases
(piglit-run twice, summary), so I have serious doubts that you actually
tested it.

If you make last minute changes to your patch, you still need to retest it.

Here is the traceback:

>> rm -rf summary; ./piglit-summary-html.py -e pass -e skip summary
results/batch-*
Traceback (most recent call last):
  File "./piglit-summary-html.py", line 98, in <module>
    main()
  File "./piglit-summary-html.py", line 94, in main
    output.generate_html(args.summaryDir, args.exclude_details)
  File "/home/kwg/Projects/piglit/framework/summary.py", line 477, in
generate_html
    exclude=exclude))
  File "/usr/lib64/python2.7/site-packages/mako/template.py", line 412,
in render
    return runtime._render(self, self.callable_, args, data)
  File "/usr/lib64/python2.7/site-packages/mako/runtime.py", line 766,
in _render
    **_kwargs_for_callable(callable_, data))
  File "/usr/lib64/python2.7/site-packages/mako/runtime.py", line 798,
in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/usr/lib64/python2.7/site-packages/mako/runtime.py", line 824,
in _exec_template
    callable_(context, *args, **kwargs)
  File
"/tmp/piglit/html-summary/home/kwg/Projects/piglit/templates/index.mako.py",
line 115, in render_body
    if line['class'] not in exclude and line['href'] is not None:
  File "/home/kwg/Projects/piglit/framework/status.py", line 139, in __eq__
    return int(self) == int(other)
ValueError: invalid literal for int() with base 10: 'skip'