[PATCH XTS 1/7] Note in README that some tests fail when run as a set.

[Peter Hutterer]

On Sat, Jul 10, 2010 at 09:41:17AM -0700, Aaron Plattner wrote:
> On Tue, Jun 29, 2010 at 04:10:19PM -0700, Peter Hutterer wrote:
> > This should be fixed in the tests themselves, but for now it's easier to
> > note this in the readme.
> > 
> > Signed-off-by: Peter Hutterer <peter.hutterer at who-t.net>
> > ---
> >  README |    9 +++++++++
> >  1 files changed, 9 insertions(+), 0 deletions(-)
> > 
> > diff --git a/README b/README
> > index 24f6f34..130cac4 100644
> > --- a/README
> > +++ b/README
> > @@ -54,3 +54,12 @@ each test program. Journal files from completed runs can be analyzed by
> >  the xts-report tool (formerly vswrpt):
> >  
> >  	$ xts-report -f /xts/output/dir/journal
> > +
> > +
> > +Bugs
> > +====
> > +Some tests provide different results when run as a set versus run
> > +one-by-one with the pt script. This has been observed with
> > +Xlib13/XGrabButton where the modifier state is not reset correctly.
> > +When debugging tests, it is recommended that failing tests are run
> > +separately to verify the result.
> 
> When we run the old-school version of XTS, the old 'xreg' script starts the
> server and then runs each test in its own server generation.  Have you
> observed any tests that PASS when run as a set that FAIL on their own?

have a look at 
http://cgit.freedesktop.org/~whot/xts-logs, there are the logs for 1.8.0
with -noreset and the logs for 1.8.0 as-is. I haven't checked for it
but it should be extractable from that data set.

Cheers,
  Peter

> I'm all for not having to do that -- server generations are pretty slow,
> and we've destroyed at least one CRT by forgetting to turn it off first.