[Piglit] Is it normal to see piglit quick.tests results changing?
idr at freedesktop.org
Thu Jan 19 08:54:34 PST 2012
On 01/14/2012 01:01 PM, Scott Tsai wrote:
> I noticed that if I run piglit quick.tests twice, some tests would
> change from pass to fail and vice versa:
> piglit-run /usr/lib64/piglit/tests/quick.tests result0
> piglit-run /usr/lib64/piglit/tests/quick.tests result1
> piglit-summary-html summary results0 results1
> xdg-open summary/index.html
> I see these tests changing:
> (... quite a bit more ...)
> Is this expected?
No, absolutely not. I've seen some of the GLX tests and time-elapsed
intermittently change state, but all of the others are rock-solid for
me. Our QA group runs piglit on Mesa every night for regression
testing, and I they see the same results I see.
Out of curiosity, do you see stable results if you run on a bare xserver
(i.e., no compositing window manager)?
> I uploaded the piglit-summary-html report to
> This is on an Intel Sandy Bridge machine running Fedora 16
> (kernel 3.1, mesa-7.11 from distro packages):
> Matěj Cepl plans to use piglit for internal QA at Red Hat and I'm
> helping him getting it packaged for Fedora. Could anyone share some
> experience on using piglit for regression testing?
More information about the Piglit