[Piglit] ctrl-c behaviour

[Eoff, Ullysses A]

Okay, so I've been messing around with some of the Python 2.6 features
for managing subprocesses and threads.  Unfortunately, those features
have presented new issues that have prevented me from successfully
satisfying this problem.  I did attempt to use the multiprocessing module,
but, it has several restrictions that I could not find a way to get around;
See http://docs.python.org/library/multiprocessing.html#multiprocessing-programming
Notably, the current state of piglit does not support picklability as
required.  And get this, the other restriction says to "Avoid terminating
processes"; WHAT!

So, I abandoned the multiprocessing module and tried another technique where
piglit explicitly tracked all the subprocesses (i.e. the ones that run the
actual test command).  Upon receiving a KeyboardInterrupt in the main thread,
piglit would call "terminate()" on one of the subprocesses (chosen using Ian's
option 4 suggestion).  Unfortunately, this appeared to be quite unstable and I
was able to crash X quite often.  Perhaps, I just needed to identify more of the
critical sections of the program and add some finer detail to control these issues.
But, currently that is beyond my level of knowledge on the proper way to use these
new features in 2.6.

But, lo and behold, I have another solution that should satisfy this use case without
resorting to Python 2.6 features.
Basically, the idea is to leave all of the GPU tests on the main thread.  It is
my understanding that the GPU tests are the ones most susceptible to hanging-up
(or at least the ones of interest). In a multithreaded Python program, the main thread
is the only thread
that receives the KeyboardInterrupt (at least on Linux).  And when subprocesses are
executing in the main thread, ctrl-c just gracefully (and perhaps magically)
interrupts that subprocess *only* and allows the rest of the python program to continue
executing as normal (this is the behavior that piglit originally exhibited).  In
the current threading model, the GPU tests are executed via a threadpool with
only 1 worker thread (since they shouldn't run concurrent to one another).  Managing
the execution of the GPU tests in a threadpool, then, seems like overkill anyway.
So moving them to the main thread only adds benefit (namely, the magic ctrl-c
support for subprocesses executed from the main thread).  What's left then is to
throw all non-gpu-bound tests into their own threadpool.  This solution, then,
supports interrupting GPU tests directly but not the CPU-only tests.  That does
not seem to be an unreasonable sacrifice, though.

Unfortunately, adding concurrency to piglit has proven to be quite a delicate and
painstaking problem since subprocesses and interrupt handling have been thrown in
the mix.  I'd love to try and come up with an optimal solution that supports everyone's
use-case, but, other priorities are demanding my attention right now.
I encourage anyone else to mess around with it if they are interested (you are a
brave soul if you do and good luck).

Patches to follow.

--U. Artie Eoff


> -----Original Message-----
> From: piglit-bounces+ullysses.a.eoff=intel.com at lists.freedesktop.org
> [mailto:piglit-bounces+ullysses.a.eoff=intel.com at lists.freedesktop.org]
> On Behalf Of Chad Versace
> Sent: Friday, February 11, 2011 3:15 PM
> To: piglit at lists.freedesktop.org
> Subject: Re: [Piglit] ctrl-c behaviour
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> I second Ian's option 4.
> 
> In Python >= 2.6, is it possible to implement option 4 in a
> platform-neutral way? If so, I think this justifies bumping the version
> on Piglit's Python requirement.
> 
> Chad Versace
> chad at chad-versace.us
> 
> - --
> Chad Versace
> chad at chad-versace.us
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
> 
> iQIcBAEBAgAGBQJNVcMQAAoJEAIvNt057x8iuUgP/AyQDbd/KVf2n/XUYy1ZH5q3
> sgYLhfdq0d4parUxFY5z0lJCYg+pCr6sX4XJdrvl/GXjPx5QmK9vadUa9Wsn8LOV
> AJnlygLfxfeImP9//tAUzSH/Wz1OrI4mhDBaYXfjIaBFSwSyAY4Q2epin2sYFu6u
> Tch8bzH3eBBZRcHlGcCIUvdFSdhdN6WcZFpjnRV7ugXJzNP1uAmeOStKLKeqj6Os
> AEXEtI8TSCykwzpHrFUAIrSjvIl7glURXNx67jzIpvhk5+JdBtbZK+jDUUPQOE2C
> nYiVKeYA+FUh9pdy4NIW1YsDU9OGwhkalyTnpeJnebkbJ1A09zVBdYA6CxPx2Pmj
> F0pbLlone/5MDKVpdMmtossQpkzbtaZxkktCD252SHdgnKX5/KuOxjkzI2+r1K+D
> yOJqKC9WeZR626D2ZKh2ox1iz4X891g/Xf4AejINZks1nRZ7krTTntMIDxh+zwbt
> Mv9axsA+j1uuWD8py53S7aAXqH72XL6g3Zq41rBukV3jPwF+wQFYpWPYNLCprBJ4
> JlAL/n1SKSYHCwsVOG1a8b8zypgLtUcJazXgKx/oA+h8udsl+mUBfwxq/n+YZRgN
> aTrv8s2wpsm6yjJl+D4u4UFMSuTmtDytjucPvWkvxGT9bQ0E45br0MUzycH298Qf
> p/bf2ER+jV6j7ugBkox9
> =9Na5
> -----END PGP SIGNATURE-----
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit