[igt-dev] [PATCH i-g-t] lib/kmod: Stop producing results at all for kernel selftests on taint

[Chris Wilson]

Quoting Petri Latvala (2021-03-02 12:39:35)
> On Tue, Mar 02, 2021 at 11:48:02AM +0000, Chris Wilson wrote:
> > Quoting Petri Latvala (2021-02-25 08:35:53)
> > > Instead of producing skips for the rest when one selftest taints the
> > > kernel, stop running them altogether. Having the skips produced yields
> > > no value and just makes future improvements (like correctly tagging
> > > tests that cause taints) harder. In effect, this gets us back to the
> > > old setup when tainting made igt_runner immediately kill the test and
> > > similarly made us not get spurious results for the rest of the
> > > selftests.
> > 
> > Skip isn't spurious here. I don't see how this impacts tagging the
> > earlier tests. So the only question for me is whether SKIP or NOTRUN is
> > more applicable. And skip is far more informative when run by hand...
> 
> When run by hand on an already tainted kernel, the first (specified)
> selftest still gets a skip and others don't get executed.
> 
> The immediate value I'll be getting from this is my ongoing work on
> the ugly igt at runner@aborted pseudoresult to instead mark the last
> executed test as 'ABORT' result when igt_runner decides to abort the
> execution (due to a taint). If we have skips, I'd need to do some
> selftest-specific handling to not mark the last executed dynamic
> subtest, but the last non-skipping dynamic subtest.

The problem does not seem to be intrinsic to igt_kselftests, as any
dynamic subtest that triggers a taint will then proceed to run the next
igt_dynamic which may pass, fail or skip. The runner may not have a
chance to check for a taint until the end of the subtest. It sounds like
a job for the igt_runner <-> igt channel.
-Chris