[Piglit] Regression with eeec306bc2197a2fc0eeb064462a1244d39c16fa "framework: don't report failed cases as PASS"

Tom Stellard tom at stellard.net
Mon Aug 26 14:50:04 PDT 2013

On Mon, Aug 26, 2013 at 11:59:40AM -0700, Dylan Baker wrote:
> Tom,
> I'm a little confused, this was the exact intention of the patch series.

My understanding was that the intention was to fix incorrect reporting
when multiple subtests had the same name.  For example, with the old
behavior, this:

PIGLIT:subtest {'+neg' : 'fail'}
PIGLIT:subtest {'+neg' : 'pass'}

The '+neg' subtest would be displayed as 'pass' in the summary, because
the last subtest result was overwriting the first.  I thought the patch
would fix this, so that '+neg' would display as 'fail' in the summary.

I think the patches fixes this, but it also introduces another problem
with results like this:

PIGLIT:subtest {'-pos' : 'pass'}
PIGLIT:subtest {'+inf' : 'fail'}

In this case the summary shows both '-pos' and '+inf' as failed, which
it shouldn't.  If this really was the intention of the patch, then I
think it should be reverted, because it completed defeats the purpose
of having subtests.


> Dylan
> On Monday 26 August 2013 10:09:10 Tom Stellard wrote:
> > Hi Homer,
> > 
> > This commit introduces a regression in piglit for reporting subtest
> > results.  Now if one subtests fails, all subtests report failure, which
> > is incorrect.  I've looked through the changes in this commit, and I
> > don't understand which change introduced the regression, could you take
> > a look?
> > 
> > Thanks,
> > Tom
> > _______________________________________________
> > Piglit mailing list
> > Piglit at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/piglit

More information about the Piglit mailing list