[Piglit] [PATCH] framework/run.py: allow additional excluded tests on resume
Mason, Michael W
michael.w.mason at intel.com
Tue May 19 09:45:59 PDT 2015
> -----Original Message-----
> From: Dylan Baker [mailto:baker.dylan.c at gmail.com]
> Sent: Tuesday, May 19, 2015 9:42 AM
> To: Daniel Vetter
> Cc: Mason, Michael W; piglit at lists.freedesktop.org
> Subject: Re: [Piglit] [PATCH] framework/run.py: allow additional excluded tests on resume
> On Tue, May 19, 2015 at 12:06:54PM +0200, Daniel Vetter wrote:
> > On Mon, May 18, 2015 at 05:12:17PM -0700, Dylan Baker wrote:
> > > I'm not sure this is a good idea,
> > >
> > > Ken Graunke and I talked this to death when I did the refactor that
> > > split the run and resume code into separate paths, and we agreed
> > > that changing the tests during a resume would make it impossible to
> > > reproduce a run, since the regex you pass might exclude tests that
> > > have already completed.
> > >
> > > That said, I'm opened to being convinced it is a good idea.
> > With your patches to mark a test as incomplete, should we instead just
> > have an option on resume to mark all incomplete tests as failed and
> > not try to restart them? Or is that already what happens (tbh I
> > haven't tried).
> > -Daniel
> Currently they're always retried. It seems reasonable to me to either change the behavior or add a switch to change the
> behavior to either mark them as failed or just leave them as incomplete.
How about something like this instead of my original patch? I think we should preserve the fact that some tests didn't complete. I'll submit this in a separate email if you all agree.
@@ -313,6 +313,9 @@ def resume(input_):
help="Optionally specify a piglit config file to use. "
"Default is piglit.conf")
+ parser.add_argument("-n", "--no-retry", dest='no_retry',
+ help="Do not retry incomplete tests")
args = parser.parse_args(input_)
@@ -342,7 +345,7 @@ def resume(input_):
# Don't re-run tests that have already completed, incomplete status tests
# have obviously not completed.
for name, result in results.tests.iteritems():
- if result['result'] != 'incomplete':
+ if result['result'] != 'incomplete' or args.no_retry:
More information about the Piglit