✗ CI.xeBAT: failure for lib/kunit: Support writable filter* parameters of kunit module (rev2)

[Janusz Krzysztofik]

On Thursday, 1 February 2024 15:13:31 CET Lucas De Marchi wrote:
> On Thu, Feb 01, 2024 at 12:06:35PM +0100, Janusz Krzysztofik wrote:
> >On Wednesday, 31 January 2024 21:51:15 CET Lucas De Marchi wrote:
> >> On Wed, Jan 31, 2024 at 08:22:35PM +0100, Janusz Krzysztofik wrote:
> >> >On Wednesday, 31 January 2024 19:36:57 CET Patchwork wrote:
> >> >> == Series Details ==
> >> >>
> >> >> Series: lib/kunit: Support writable filter* parameters of kunit module
> >> >(rev2)
> >> >> URL   : https://patchwork.freedesktop.org/series/129174/
> >> >> State : failure
> >> >>
> >> >> == Summary ==
> >> >>
> >> >> CI Bug Log - changes from XEIGT_7699_BAT -> XEIGTPW_10613_BAT
> >> >> ====================================================
> >> >>
> >> >> Summary
> >> >> -------
> >> >>
> >> >>   **FAILURE**
> >> >>
> >> >>   Serious unknown changes coming with XEIGTPW_10613_BAT absolutely need to
> >> >be
> >> >>   verified manually.
> >> >>
> >> >>   If you think the reported changes have nothing to do with the changes
> >> >>   introduced in XEIGTPW_10613_BAT, please notify your bug team (I915-ci-
> >> >infra at lists.freedesktop.org) to allow them
> >> >>   to document this new failure mode, which will reduce false positives in
> >> >CI.
> >> >>
> >> >>
> >> >>
> >> >> Participating hosts (4 -> 4)
> >> >> ------------------------------
> >> >>
> >> >>   No changes in participating hosts
> >> >>
> >> >> Possible new issues
> >> >> -------------------
> >> >>
> >> >>   Here are the unknown changes that may have been introduced in
> >> >XEIGTPW_10613_BAT:
> >> >>
> >> >> ### IGT changes ###
> >> >>
> >> >> #### Possible regressions ####
> >> >>
> >> >>   * igt at xe_live_ktest@bo:
> >> >>     - bat-adlp-7:         [PASS][1] -> [SKIP][2] +2 other tests skip
> >> >>    [1]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7699/bat-adlp-7/
> >> >igt at xe_live_ktest@bo.html
> >> >>    [2]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_10613/bat-adlp-7/
> >> >igt at xe_live_ktest@bo.html
> >> >
> >> >Hi Lucas,
> >> >
> >> >We've got an empty list of test cases because the current algorithm of IGT
> >> >KUnit implementation expects subtest name provided via struct kunit_tests
> >> >to be the same as name of a test suite provided by the KUnit test module.  As
> >> >a special case, if the module provides only one test suite (most do) and the
> >> >name of it is the same as the module name with the _test or _kunit suffix
> >> >truncated (also the most common case) then subtest name may be omitted (i.e.,
> >> >.name field may be NULL).
> >> >
> >> >I have to document the above, but we also have to use the same names on both
> >> >sides (KUnit test module and its corresponding IGT test).  Now we don't, e.g.
> >> >test suite named "xe_bo" is provided by xe_bo_test module while subtest name
> >> >"bo" is specified for that module in IGT xe_live_ktest.
> >>
> >> in the end we want to have only one module: xe_live_test.ko and then
> >> filter out by the testunit. From what I've seen previously in igt, it
> >> would be a matter of always reloading the same module, but passing a
> >> different name.
> >
> >As long as the module will provide several test suites, each of those test
> >suites can be mapped to a separate subtest with the same name as the test
> >suite, and with its test cases discovered and executed automatically by IGT as
> >dynamic sub-subtests, still selectable individually via IGT --dynamic command
> >line option.  For that to work this way, populating .name fields of the struct
> >kunit_tests table with subtest names that are the same as test suite names is
> >mandatory, otherwise IGT will expose one subtest, named after the module name,
> >with dynamic sub-subtests named <suite_name>-<case_name>.
> 
> that's what I did when I submitted the module rename:
> https://lore.kernel.org/igt-dev/20231205224926.2188996-1-lucas.demarchi@intel.com/
> 
> This one is to be applied on top of these fixes to kunit handling.

Do your subtest names match test suite names used in xe_live_test.ko?  AFAICS, 
https://patchwork.freedesktop.org/series/126786/ referred in the IGT patch 
you've pointed out didn't touch test suite names while combining source files 
of several now separate modules into that one module, and test suite names 
still reported by those separate modules don't match -- they start with xe_ 
prefix.  We need to have those mismatches fixed on one side or another.

Thanks,
Janusz

> 
> Lucas De Marchi
>