[Intel-gfx] [PATCH i-g-t v2] lib/igt_core.c: Expand --run-subtest functionality.

[Dave Gordon]

On 28/01/16 11:05, Derek Morton wrote:
> Added support for specifying arbitary lists of subtests to run, or
> to exclude from being run if prefixed by ^ or !.
>
> subtest1,subtest2 Will run subtest1 and subtest2
> ^subtest1,subtest2 or !subtest1,subtest2 will run all subtests except
> subtest1 and subtest2.
>
> Any subtest string not starting ! or ^ and not containing a comma is
> treated as a normal wildcard expression.
>
> This is required mainly on android to exclude subtests that test
> features that do not exist in the android driver while still being able
> to run other subtests in the binary when a wildcard expression is
> insufficient.
>
> v2: Use comma as list separator (Ville Syrjala)
> support both ^ and ! as not operators (Dave Gordon)
>
> Signed-off-by: Derek Morton <derek.j.morton at intel.com>
> ---
>   lib/igt_core.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++--
>   1 file changed, 46 insertions(+), 2 deletions(-)
>
> diff --git a/lib/igt_core.c b/lib/igt_core.c
> index 6b69bb7..5d243af 100644
> --- a/lib/igt_core.c
> +++ b/lib/igt_core.c
> @@ -207,7 +207,15 @@
>    * To do that obtain the lists of subtests with "--list-subtests", which can be
>    * run as non-root and doesn't require the i915 driver to be loaded (or any
>    * intel gpu to be present). Then individual subtests can be run with
> - * "--run-subtest". Usage help for tests with subtests can be obtained with the
> + * "--run-subtest". --run-subtest accepts wildcard characters. A list of
> + * subtests to run may be specified by using ',' as a separator.
> + * A prefix of ^ or ! may be added to invert the logic, e.g. run all tests except...
> + *
> + * - --run-subtest basic* will run all subtests starting basic.
> + * - --run-subtest subtest1,subtest2 will run only subtest1 and subtest2
> + * - --run-subtest ^subtest1,subtest2 will run all those except subtest1 and subtest2
> + *
> + * Usage help for tests with subtests can be obtained with the
>    * "--help" command line option.
>    */

This is the syntax option 1 from my previous mail (but not allowing 
multiple --run-subtest options?), which I think is a bit less powerful 
and a bit less desirable because of the perhaps unexpected precedence of 
OR above NOT (^sub1,sub2 => NOT(sub1 OR sub2).

Do you think the advantages of option 2 (full generality, can express 
any boolean, expected precedence of operators) would make it worth the 
extra effort?

Whichever we pick now, it will be set in stone as it will be very 
inconvenient to edit every script using the feature once it's been in 
use for a while!

Another idea: have you considered bash(1) extglob syntax? See the 
manpage for bash, section "Pattern Matching". It's quite powerful, 
familiar to least to script writers, and easy to try out in bash:

$ : enable extglob mode
$ shopt -s extglob
$ : create an array of test names
$ declare -a tests=( starting phase1 phase2 wibble phase3 wobble done )
$ : see what matches a pattern:
$ function match() { for w in "${tests[@]}" ; do [[ "$w" == $1 ]] && 
echo "$w" ; done ; }
$ match '*'
starting
phase1
phase2
wibble
phase3
wobble
done
$ match 'phase*'
phase1
phase2
phase3
$ match 'w?bble'
wibble
wobble
$ match '[!pw]*'
starting
done
$ match '@(starting|done)'
starting
done
$ match '!(starting|done)'
phase1
phase2
wibble
phase3
wobble
$ match '!(phase*)'
starting
wibble
wobble
done
$ match '!(phase*|w?bble)'
starting
done
$ : etc, etc, ad lib

.Dave.