[igt-dev] [PATCH i-g-t] lib: Update selftests to use dynamic subtests

[Petri Latvala]

On Fri, Feb 14, 2020 at 09:48:29PM +0000, Chris Wilson wrote:
> Use the new igt_subtest_with_dynamic to nicely group the dynamic
> subtests together.
> 
> Signed-off-by: Chris Wilson <chris at chris-wilson.co.uk>
> Cc: Petri Latvala <petri.latvala at intel.com>
> Cc: Tomi Sarvela <tomi.p.sarvela at intel.com>
> ---
>  lib/igt_kmod.c | 23 +++++++++++++++++++----
>  1 file changed, 19 insertions(+), 4 deletions(-)
> 
> diff --git a/lib/igt_kmod.c b/lib/igt_kmod.c
> index 05019c24c..e701545d3 100644
> --- a/lib/igt_kmod.c
> +++ b/lib/igt_kmod.c
> @@ -21,6 +21,7 @@
>   * IN THE SOFTWARE.
>   */
>  
> +#include <ctype.h>
>  #include <signal.h>
>  #include <errno.h>
>  
> @@ -602,6 +603,18 @@ void igt_kselftest_fini(struct igt_kselftest *tst)
>  	kmod_module_unref(tst->kmod);
>  }
>  
> +static const char *unfilter(const char *filter, const char *name)
> +{
> +	if (!filter)
> +		return name;
> +
> +	name += strlen(filter);
> +	if (!isalpha(*name))
> +		name++;
> +
> +	return name;
> +}
> +
>  void igt_kselftests(const char *module_name,
>  		    const char *options,
>  		    const char *result,
> @@ -618,10 +631,12 @@ void igt_kselftests(const char *module_name,
>  		igt_require(igt_kselftest_begin(&tst) == 0);
>  
>  	igt_kselftest_get_tests(tst.kmod, filter, &tests);
> -	igt_list_for_each_entry_safe(tl, tn, &tests, link) {
> -		igt_subtest_f("%s", tl->name)
> -			igt_kselftest_execute(&tst, tl, options, result);
> -		free(tl);
> +	igt_subtest_with_dynamic(filter ?: "all") {
> +		igt_list_for_each_entry_safe(tl, tn, &tests, link) {
> +			igt_dynamic_f("%s", unfilter(filter, tl->name))
> +				igt_kselftest_execute(&tst, tl, options, result);
> +			free(tl);
> +		}
>  	}



Thanks for starting this, exactly what dynamic subtests are for.

Pre-acked-by: Petri Latvala <petri.latvala at intel.com>

CI side should be made ready before landing this. With kernel
selftests being now launched from statically enumerable subtests,
selftests can be added to fast-feedback.testlist and un-blacklisted
from shards where applicable.

Tomi, what do we run (and what do we want to) in BAT now? Objections
to running selftests in shards mixed in with other tests?

Runtime is a concern if all (FSVO all) selftests are launched with one
entry in a testlist. What is the total runtime on a random platform
for, say, i915_selftest subtests (post patch)? We might need to split
more than just live/mock.


-- 
Petri Latvala