[Ezbench-dev] [PATCH 00/25] runner.sh and robustness improvements

[Petri Latvala]

This series contains the runner.sh commits at the time of sending,
plus some improvements I have made when testing with igt
runs. Martin's 18 commits are unchanged from what was in git at the
time.

My commits increase ezbench robustness when igt encounters severe and
weird kernel bugs.

 - Keep compiling if graphical output cannot be shown
 - Use a hardware watchdog when running tests
 - Handle testing failures and test run aborts in several
   places. Meaning cases where the machine isn't jamming enough to
   cause a watchdog to reboot, but when testing just should abort, no
   results can be trusted if any more tests are run without rebooting
   first, and after rebooting testing should resume properly. This is
   a tricky area and stuff might still be missing.

Tested with icky IGT tests on gloriously broken kernels, and working
fine for me so far.


Martin Peres (18):
  profile: introduce a helper to compare versions reliably
  profile/repos: add profile_repo_version_title
  profile/git-repo: fix profile_get_built_versions for the first run
  runner.sh: introduce a new leaner and more modular runner
  smartezbench: make use of the new runner
  report/journal: move the timestamp inside a attribute dict
  report/journal: parse the "test(ed)" ops differently
  journal: add the run's output file for the "test" and "tested" output
  report/journal: detect incomplete test runs
  test/piglit: allow resuming incomplete runs
  test/piglit: do not report "completed" if the report generation failed
  test/igt: add resume support
  smartezbench: resume incomplete runs, if possible
  test/piglit: allow keeping piglit/igt reports
  smartezbench: improve the output of TaskEntry
  report: use the new runner
  scm/ResultsDAG: show the number of commits between versions in the
    .dot
  smartezbench/schedule_enhancement: print the task list in the debug
    log

Petri Latvala (7):
  kernel-igt/compile.conf: Use timeout with chvt
  report.py: Mark a run as incomplete if "tested" entry is entirely
    missing.
  utils/owatch: Overwatch, a watchdog wrapper
  runner.sh: Don't mark test as completed unless it signals as completed
  tests/piglit/igt: Use owatch, reboot on monitored abort
  runner.py: Signal reboot required on runner error
  .gitignore: Ignore Emacs backup files (*~)

 .gitignore                                |   1 +
 README                                    |   6 +-
 core.sh                                   |   8 +-
 profiles.d/kernel-igt/conf.d/compile.conf |   2 +-
 profiles.d/utils/auto-deploy.sh           |   4 +-
 profiles.d/utils/common.sh                |  13 +-
 profiles.d/utils/git-repo.sh              |  14 +-
 profiles.d/utils/no-repo.sh               |   7 +
 python-modules/ezbench/report.py          |  67 ++-
 python-modules/ezbench/runner.py          | 233 +++++++++
 python-modules/ezbench/scm.py             |   8 +-
 python-modules/ezbench/smartezbench.py    | 272 ++++++----
 runner.sh                                 | 802 ++++++++++++++++++++++++++++++
 stats/compare_reports.py                  |   2 +-
 tests.d/piglit/igt.test                   | 108 ++--
 tests.d/piglit/piglit.test                |  69 ++-
 unit_tests/ez_ut.py                       |   2 +-
 user_parameters.sh.sample                 |   1 +
 utils/owatch/.gitignore                   |   2 +
 utils/owatch/Makefile                     |   8 +
 utils/owatch/owatch.c                     | 258 ++++++++++
 21 files changed, 1690 insertions(+), 197 deletions(-)
 create mode 100755 runner.sh
 create mode 100644 utils/owatch/.gitignore
 create mode 100644 utils/owatch/Makefile
 create mode 100644 utils/owatch/owatch.c

-- 
2.9.3