<html> <head> <base href="https://bugs.freedesktop.org/"> </head> <body> <div> <a class="bz_bug_link bz_status_NEW " title="NEW - When testlist contains non-existing (removed) test, piglit excepts out" href="https://bugs.freedesktop.org/show_bug.cgi?id=99649#c3">Comment # 3</a> on <a class="bz_bug_link bz_status_NEW " title="NEW - When testlist contains non-existing (removed) test, piglit excepts out" href="https://bugs.freedesktop.org/show_bug.cgi?id=99649">bug 99649</a> from <a class="email" href="mailto:tomi.p.sarvela@intel.com" title="Tomi Sarvela <tomi.p.sarvela@intel.com>"> Tomi Sarvela</a> <pre>I was asked to explain my issue a bit better, so here's by pov: Catching the exception is clear. The issue is known and easily hit, and non-catched exception is a bug. Choosing what to do: Continue with Warning - This helps CI: testlist might be maintained locally, and removing test upstream breaks every run after update at the moment - This helps user: warning about missing test, but test anyway what you can - Test order is in meta/tests, just as with regexes Abort with Error - User: makes it clear that testlist can't be run as-is. Force to edit testlist - CI: breaks. Resolution: don't build piglit from upstream automatically - Test order can be read from testlist, in addition to meta/tests One possibility is to choose the behaviour with a commandline option. I wouldn't mind that, either. I don't think testlist are very widely used yet, so choosing the correct default is possible. I'm liking "Continue with Warning", because that's the unix way.</pre> </div> <hr> You are receiving this mail because: <ul> <li>You are the QA Contact for the bug.</li> </ul> </body> </html>