[Piglit] [PATCH v2] core.py: fix subtest json reporting

Tom Stellard tom at stellard.net
Thu Sep 5 11:34:26 PDT 2013


For now, I've revert the commit that introduces this regression.

On Fri, Aug 30, 2013 at 07:39:28AM -0700, Dylan Baker wrote:
> I have sent out a v3 of this patch, which I believe addresses the issues 
> raised.
> 

I tested v3, but I'm still seeing the problem where if one
subtest fails then piglit reports all the subtests as 'fail'.  I've attached
an example results file in case that helps.

-Tom

> On Thursday 29 August 2013 17:54:57 Eric Anholt wrote:
> > Tom Stellard <tom at stellard.net> writes:
> > > On Wed, Aug 28, 2013 at 12:35:02PM -0700, Dylan Baker wrote:
> > >> Previously every subtest wrote an entry in the file as a full test, this
> > >> is a bug since the fake-fulltest (actually subtest) reported with the
> > >> full test value, not it's own value.
> > >> 
> > >> This patch changes the behvaior so that a test with subtests only gets
> > >> one entry, with the full test result value as it's value.
> > > 
> > > I would really like the old behavior preserved.  If each subtest doesn't
> > > have its own entry in the summary, then there is really no point to
> > > having subtests.
> > > 
> > > I am having a little trouble following the code, so here are some examples
> > > of the past and current behavior to help clarify the situation.
> > > 
> > > Lets say we have a Test A with subtests:
> > > 1 => 'FAIL'
> > > 2 => 'PASS'
> > > 3 => 'FAIL'
> > > 
> > > The original way of displaying this (i.e before
> > > eeec306bc2197a2fc0eeb064462a1244d39c16fa) was:
> > > 
> > > A     2/3
> > > 1     FAIL
> > > 2     PASS
> > > 3     FAIL
> > > 
> > > The way it is displayed with the current code from git is:
> > > 
> > > A    0/3
> > > 1    FAIL
> > > 2    FAIL
> > > 3    FAIL
> > > 
> > > The way it is displayed with this patch is:
> > > 
> > > A    FAIL
> > > 
> > > The current state of things is making it difficult to compare subtest
> > > results. Would anyone object to me reverting
> > > eeec306bc2197a2fc0eeb064462a1244d39c16fa while we sort out how to best
> > > handle subtests?
> > 
> > Yeah, it should definitely be backed out.
> 


More information about the Piglit mailing list