[PATCH i-g-t 0/1] relax doc generation for non-compiled tests

[Manszewski, Christoph]

Hi Kamil,

On 12.09.2024 16:04, Kamil Konieczny wrote:
> New tests added with conditional compilation result in error reported
> when generating testlist. For example, with a recent series with new
> eudebug igt tests from here:
> https://patchwork.freedesktop.org/series/136623/
> ("Test coverage for GPU debug support")
> 
> when --exclude-files option was removed from version 6:
> https://patchwork.freedesktop.org/series/136623/#rev6
> 
> then compiling with testplan report warnings:
> 
> xe_eudebug.testlist: Testlist file not found.
> xe_eudebug_online.testlist: Testlist file not found.
> xe_exec_sip_eudebug.testlist: Testlist file not found.
> Warning: Documented igt at xe_eudebug@attach-debug-metadata doesn't exist on source files
> Warning: Documented igt at xe_eudebug@basic-client doesn't exist on source files
> Warning: Documented igt at xe_eudebug@basic-client-th doesn't exist on source files
> [...rest omitted...]
> 
> and exits with an error.
> 
> Relax that check to only report error on missing documentation, and treat
> new added tests without a compiled binary as a new tests under development
> or as tests outside a compilation scope (such as chamelium). This should
> allow to have new tests in documentation and track them.
> 
> With dropped --exclude-files patch and dropped changes in docs/testplan/meson.build
> generation of testplan succeeds and in report it can be find:

Well the only issue I see with this approach is that we actually 
generate documentation for tests that weren't built, while the 
'--exclude_files' flag skips doc generation all together. But if this is 
intended/accepted I am happy to move forward with your solution.

Thanks,
Christoph

> 
> [1897/1905] Generating docs/testplan/xe_tests.rst with a custom command
> xe_eudebug.testlist: Testlist file not found.
> xe_eudebug_online.testlist: Testlist file not found.
> xe_exec_sip_eudebug.testlist: Testlist file not found.
> xe_eudebug: No binary found, test was not compiled.
> xe_eudebug_online: No binary found, test was not compiled.
> xe_exec_sip_eudebug: No binary found, test was not compiled.
> Info: Documented igt at xe_eudebug@attach-debug-metadata don't have a binary xe_eudebug
> Info: Documented igt at xe_eudebug_online@basic-breakpoint don't have a binary xe_eudebug_online
> Info: Documented igt at xe_exec_sip_eudebug@breakpoint-waitsip don't have a binary xe_exec_sip_eudebug
> [1898/1905] Generating docs/testplan/i915_tests.rst with a custom command
> [1899/1905] Generating docs/testplan/kms_tests.rst with a custom command
> [1900/1905] Generating docs/testplan/intel-ci-tests with a custom command
> [1901/1905] Generating docs/testplan/xe_tests.html with a custom command
> [1902/1905] Generating docs/testplan/i915_tests.html with a custom command
> [1903/1905] Compiling C object runner/runner_test.p/runner_tests.c.o
> [1904/1905] Generating docs/testplan/kms_tests.html with a custom command
> [1905/1905] Linking target runner/runner_test
> 
> 
> Cc: Christoph Manszewski <christoph.manszewski at intel.com>
> Cc: Zbigniew Kempczyński <zbigniew.kempczynski at intel.com>
> Cc: Jari Tahvanainen <jari.tahvanainen at intel.com>
> Cc: Katarzyna Piecielska <katarzyna.piecielska at intel.com>
> Cc: Mauro Carvalho Chehab <mchehab at kernel.org>
> 
> Kamil Konieczny (1):
>    scripts/test_list: relax treatment of non-compiled tests
> 
>   scripts/test_list.py | 31 +++++++++++++++++++++++++++++--
>   1 file changed, 29 insertions(+), 2 deletions(-)
>