[Piglit] Old results, subtests, and piglit-summary-html

[Dylan Baker]

On Tuesday, May 27, 2014 12:29:08 Ilia Mirkin wrote:
> Hello,
> 
> I'm seeing an issue where if I use "old" piglit results (I think
> something was done to the subtest logic), I see an explosion of
> results. I see
> 
> ARB_texture_rg
>  -> texwrap formats-int bordercolor-swizzled
>     -> GL_R16I, swizzled, border color only
>       -> GL_R16I, swizzled, border color only
> 
> Note the duplication. This causes an N^2 effect with subtests, which
> is rather unfortunate... the number of tests becomes like 100K instead
> of 10K. Here is a directory with a whole bunch of old results:
> http://people.freedesktop.org/~imirkin/piglit/
> 
> Take any one of them and run it through piglit-summary-html and you'll
> see what I mean. I haven't really done more investigation, just wanted
> to raise the issue. However I suspect that commit
> 
http://cgit.freedesktop.org/piglit/commit/?id=80364ef8d6cc2ff8b3910aab4
402305ab1aeb0f2
> may be at fault (and
> 
http://cgit.freedesktop.org/piglit/commit/?id=3624ae03cfef74522e3fecd36
2f691753f32fed5
> removed the extra duplication for new results, but that doesn't fix
> the existing ones).
> 
> Cheers,
> 
>   -ilia

Yes, this is an unfortunate side effect of fixing the subtests logic in the 
summary handling.

Previously summary had no idea what subtests where, and so as a hack 
each test with a subtest would write n*subtests into the json. This 
ballooned our json output 60%. I introduced logic to handle subtests some 
time ago, but there were a couple of bugs that I couldn't solve and so I 
reverted it. A couple months ago I sent out patches to re-enable it, I had 
since fixed the bugs.

The problem is that since the old tests contain n*s entries, and each 
subtest is enumerated, you end up with a lot of summary entries in old 
results.

I will point out that these patches were on the list for more than a month 
unreviewed, and I bumped them several times before pushing them.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20140527/f4abdbf0/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20140527/f4abdbf0/attachment.sig>