[PATCH i-g-t v13 0/3] igt_facts for fact tracking

[Peter Senna Tschudin]

igt_facts is a library that tracks facts about the system and reports
changes to the facts. This is useful for tracking changes in the
system configuration that may affect the test results.

The patch series adds the library, unit testing, the lsfacts tool, and
make changes to igt_runner to use the library. For igt_runner the facts
are disabled by default. To enable them use the command line argument
-f or --facts.

Here's 3 reasons why igt_facts seem useful:

  - CI itself performs some of these checks in the pipelines (loaded
    modules, kernel taints before testing starts, likely more in the
    future), so this could simplify fetching this data if we just
    parsed the IGT lsfacts output.

  - Simplicity: allows for quick examination of runner logs,
    identifying successful tests and isolating those that left the
    system tainted or caused the GPU to drop off the bus. This enables
    efficient filtering of relevant logs to pinpoint issues near
    reported fact changes. While this information could also be derived
    from dmesg outputs or standard error logs, igt_facts is more
    straightforward and convenient, particularly for non-kernel
    developers tasked with review and debugging.

  - It also makes it easier to notice passing tests that leave the
    system in an unclean state.

Here is an example of the output of igt_runner when using igt_facts:

  [229.606139] [FACT before any test] new: hardware.pci.gpu_at_addr.0000:03:00.0: 8086:e20b Intel Battlemage (Gen20)
  [229.606305] [FACT before any test] new: kernel.is_tainted.taint_warn: true
  [229.608841] [001/267] (600s left) xe_module_load (load)
  [229.641224] Starting subtest: load
  [230.613328] Subtest load: SUCCESS (0.973s)
  [230.678868] [FACT xe_module_load (load)] new: hardware.pci.drm_card_at_addr.0000:03:00.0: card0
  [230.680801] [FACT xe_module_load (load)] new: kernel.kmod_is_loaded.xe: true

v13:
 - remove enabled from igt_facts_config
 - use settings->facts instead of igt_facts_config.enabled
 - update serialize_settings() and read_settings_from_file() to save
   and restore igt_runner settings to and from disk
 - update igt_runner unit testing to test that:
   - Facts are disabled by default
   - Facts can be enabled by command line arguments
   - The choice about facts being enabled or not is saved to disk
     and restored from disk

v12:
 - split the patch in 3
 - updated comment style on .h files
 - updated module list to be closer to lib/drmtest.c and to
   include a comment mentioning lib/drmtest.c
 - Added a configuration struct to track the command line argument
   and udev status.
 - Add mechanism to disable udev to prevent error message spamming
   when udev is not available.
 - runner/executor: moved the call to igt_facts_lists_init() to after
   dry run check.
 - moved variable definitions from igt_facts.h to igt_facts.c
 - added #ifndef guards to igt_facts.h
 - removed double new lines
 - updated comment style that were still using //

v11:
 - fix typo

v10:
 - fix memory leaks from asprintf (Thank you Dominik Karol!)
 - fix comments for consistency (Thank you Dominik Karol!)

v9:
 - do not report new hardware when loading/unloading kmod changes the
   string of the GPU name. I accidentally reintroduced this issue
   when refactoring to use linked lists.
 - add tools/lsfacts: 9 lines of code that print either the facts or
   that no facts were found.
 - fix code comments describing functions
 - fix white space issues

v8:
 - fix white space issues

v7:
 - refactor to use linked lists provided by igt_lists
 - Added function arguments to code comments
 - updated commit message

v6:
 - sort includes in igt_facts.c alphabetically
 - add facts for kernel taints using igt_kernel_tainted() and
   igt_explain_taints()

v5:
 - fix the broken patch format from v4

v4:
 - fix a bug on delete_fact()
 - drop glib and calls to g_ functions
 - change commit message to indicate that report only on fact changes
 - use consistent format for reporting changes
 - fix SPDX header format

v3:
 - refreshed commit message
 - changed format SPDX string
 - removed license text
 - replace last_test assignment when null by two ternary operators
 - added function descriptions following example found elsewhere in
   the code
 - added igt_assert to catch failures to realloc()

v2:
 - add lib/tests/igt_facts.c for basic unit testing
 - bugfix: do not report a new gpu when the driver changes the gpu name
 - bugfix: do not report the pci_id twice on the gpu name

CC: Helen Koike <helen.koike at collabora.com>
CC: Jani Nikula <jani.nikula at linux.intel.com>
CC: Jani Saarinen <jani.saarinen at intel.com>
CC: Janusz Krzysztofik <janusz.krzysztofik at linux.intel.com>
CC: Juha-Pekka Heikkila <juha-pekka.heikkila at intel.com>
CC: Kamil Konieczny <kamil.konieczny at linux.intel.com>
CC: Lucas De Marchi <lucas.demarchi at intel.com>
CC: Maíra Canal <mcanal at igalia.com>
CC: Melissa Wen <mwen at igalia.com>
CC: Petri Latvala <adrinael at adrinael.net>
CC: Rob Clark <robdclark at chromium.org>
CC: Ryszard Knop <ryszard.knop at intel.com>
CC: Swati Sharma <swati2.sharma at intel.com>
CC: Zbigniew Kempczyński <zbigniew.kempczynski at intel.com>
CC: dominik.karol.piatkowski at intel.com
CC: himal.prasad.ghimiray at intel.com
CC: igt-dev at lists.freedesktop.org <igt-dev at lists.freedesktop.org>
CC: katarzyna.piecielska at intel.com
CC: luciano.coelho at intel.com
CC: nirmoy.das at intel.com
CC: stuart.summers at intel.com

Peter Senna Tschudin (3):
  lib/igt_facts: Library and unit testing for fact tracking
  tools/lsfacts: Add tool for listing facts
  runner/executor: Integrate igt_facts functionality

 lib/igt_facts.c       | 791 ++++++++++++++++++++++++++++++++++++++++++
 lib/igt_facts.h       |  47 +++
 lib/meson.build       |   1 +
 lib/tests/igt_facts.c |  18 +
 lib/tests/meson.build |   1 +
 runner/executor.c     |  14 +
 runner/runner_tests.c |  11 +-
 runner/settings.c     |  10 +-
 runner/settings.h     |   1 +
 tools/lsfacts.c       |  27 ++
 tools/meson.build     |   1 +
 11 files changed, 920 insertions(+), 2 deletions(-)
 create mode 100644 lib/igt_facts.c
 create mode 100644 lib/igt_facts.h
 create mode 100644 lib/tests/igt_facts.c
 create mode 100644 tools/lsfacts.c

-- 
2.34.1