[igt-dev] [PATCH i-g-t 1/1] scripts: add a parser to produce documentation from Kernel C file metatags

[Mauro Carvalho Chehab]

On Fri, 03 Feb 2023 12:23:52 +0200
Jani Nikula <jani.nikula at linux.intel.com> wrote:

> On Fri, 03 Feb 2023, Mauro Carvalho Chehab <mauro.chehab at linux.intel.com> wrote:
> > From: Mauro Carvalho Chehab <mchehab at kernel.org>
> >
> > On a similar approach to Kernel's kernel-doc script, add a parser
> > that will produce documentation from special-purpose documentation
> > tags inside IGT tests.  
> 
> kernel-doc is really not a great model to duplicate elsewhere.

Why not? Kernel has been succeed keeping its interfaces documented. Ok,
there would be other alternatives to be used, but the main point is really
to keep the documentation as close as possible to the source code, in a
parseable way. 

> Why hand craft another C parser? There's a cost to maintaining this, and
> it's not helped by the choice of implementation language.

There is always a cost of maintaining documentation, and a cost of not
having them.

One of the biggest costs we have is related to the time for CI to run
IGT tests. The more tests we add, the higher the costs increase. That's
by far the most relevant cost here.

Right now, there are tests written several years ago, that people don't
have a clear idea why they're doing, and if they should belong to a
basic acceptance criteria or if some other tests are already covering
the functionality of them. So, multiple tests for the same feature could
end being running as part of the test plans, increasing the costs of
running such tests (energy costs, machine costs, etc).

Having test plans documenting what's inside each test file allows to
decide what of those tests make sense to run for each new patch added
to the tree, and what of those are there just to cover some borderline
cases and won't need to be run every time.

> The script alone isn't enough; how do you bolt this into other
> documentation? More ad hoc scripting like
> docs/reference/igt-gpu-tools/{generate_description_xml.py,generate_programs_xml.sh}?

No need. This script could be called to produce, let's say, Xe documentation by adding
a target at tests/xe/meson.build that would do something similar to:

	${SRCDIR}/scripts/igt-doc ${SRCDIR}/tests/xe/*.c > ${DESTDIR}/docs/xe_tests.rst

And CI would be running:

	${SRCDIR}/scripts/igt-doc ${SRCDIR}/tests/xe/*.c --check

In order to verify if all the tests inside a given directory (in this 
example, tests/xe) are documented.

With such automation in place, if someone writes a new test or subtest
and forgets to update the test description, CI could return warning
messages, asking the developer to fill in the gaps.

Regards,
Mauro