[igt-dev] [PATCH i-g-t 00/11] runner: Don't report subtests that have dynamic subtests

[Petri Latvala]

A feature requested by Martin before full-on conversion to the use of
dynamic subtests can begin from cibuglog's perspective.

If we have dynamic subtests to report, don't also report the subtest
that contains them.

For example, if we have test "some-test" with subtest "some-subtest"
with dynamic subtests "foo" and "bar", we only report
igt at some-test@some-subtest at foo and igt at some-test@some-subtest at bar,
avoiding igt at some-test@some-subtest altogether.

Parsing dynamic subtests has been changed so that the first dynamic
subtest includes all output from the beginning of the subtest so we
get the setup messages *somewhere*. Same for the last dynamic subtest,
we include all output to the end of the subtest output.

Note that this only changes what gets into results.json; Running the
test yourself still gives you all the output.


Petri Latvala (11):
  runner/resultgen: Extract igt-version field handling to a helper
  runner/resultgen: Extract finding begin/end lines for a subtest to a
    helper
  runner/resultgen: Extract finding begin/end pointers for test output
    to helpers
  runner/resultgen: Hoist handling of dynamic subtest output to a helper
  runner/resultgen: Extrude dynamic subtest result texts
  runner/resultgen: Add support for extra validation hook in
    find_matches()
  runner/resultgen: Make subtest result line finding more robust
  runner/json_tests: Adapt to dynamic subtest result parsing
  runner/resultgen: Don't report subtest result if it has dynamic
    subtests
  runner/json_tests: Adapt to no longer reporting subtests with dynamic
    subtests
  runner/json_tests: Add test for parsing dynamic subtests with same
    name

 .../0/dmesg.txt                               |  10 +
 .../0/err.txt                                 |  10 +
 .../0/journal.txt                             |   3 +
 .../0/out.txt                                 |  11 +
 .../README.txt                                |   2 +
 .../endtime.txt                               |   1 +
 .../joblist.txt                               |   1 +
 .../metadata.txt                              |  12 +
 .../reference.json                            |  84 +++
 .../starttime.txt                             |   1 +
 .../uname.txt                                 |   1 +
 .../dynamic-subtests/reference.json           |  70 +--
 runner/resultgen.c                            | 528 +++++++++++-------
 runner/runner_json_tests.c                    |   1 +
 14 files changed, 475 insertions(+), 260 deletions(-)
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/0/dmesg.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/0/err.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/0/journal.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/0/out.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/README.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/endtime.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/joblist.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/metadata.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/reference.json
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/starttime.txt
 create mode 100644 runner/json_tests_data/dynamic-subtest-name-in-multiple-subtests/uname.txt

-- 
2.19.1