[Mesa-dev] Documenting with Sphinx

[Robert Bragg]

On Tue, Dec 13, 2016 at 11:08 PM, Rob Clark <robdclark at gmail.com> wrote:
> On Tue, Dec 13, 2016 at 5:47 PM, Eric Anholt <eric at anholt.net> wrote:
>> Jason Ekstrand <jason at jlekstrand.net> writes:
>>
>>> Hey All,
>>> I don't figure this will be terribly controversial (I'm about to be wrong,
>>> aren't I?) but how do people feel about switching our "primary"
>>> documentation focus, as far as we have one, to sphinx?  Right now, Gallium
>>> uses sphinx for documenting all of it's "public" interfaces.  A while ago,
>>> Connor wrote some nice NIR documentation using sphinx that never got
>>> merged.  I've also spent some time this year and written some additional
>>> documentation about Intel surface layout that I'd like to have a place to
>>> put.  The thing that's in common with each of these is that they contain
>>> quite a bit of prose about the general theory of how things work and not
>>> just code comments so sphinx seems like a fairly good fit.  There are also
>>> some sort-of "architectural" things in the Vulkan driver that I would like
>>> to document better and we don't have a good place for that right now.
>>>
>>> Here's what I have in mind:
>>>
>>> Each component that we care to document (this may not be everything!) would
>>> have a docs/ or sphinx/ folder.  In that folder would be any side-band (not
>>> in code comments) documentation.  We would also set up some sort of
>>> source-scraping tool to turn inline comments into sphinx documentation.  At
>>> the component maintainer's discretion, that code could either be imported
>>> into sphinx whole-sale or curated by importing bits into other
>>> documentation files in a more carefully curated manner.  Of course, a
>>> maintainer could also choose to do all of their documentation side-band
>>> (sphinx allows for this) or to not document at all. :-)
>
> So, +100 to sphinx... I've been quite happy with it (and would be nice
> to merge the NIR docs too while where at it).  Maybe someday we'll
> have a "mesa book" after all ;-)
>
>
>> I've written doxygen-style comments in my code for a long time, but
>> never really looked at the doxygen output, so I wouldn't say I'm wedded
>> to doxygen.  I think some side-band docs for architectural descriptions
>> (like NIR needs) would be great, and if sphinx would help with
>> encouraging those to be written and merged, then that's pretty good with
>> me.
>>
>> I'm a little skeptical of sphinx due to not being able to extract docs
>> From C comments in code in the base tool.  Apparently Breathe can chain
>> doxygen into sphinx, so hopefully a "let's get excited about sphinx"
>> patch series would convert our doxygen-based docs generation to using
>> that.
>
> So as far as I understand on kernel side we are *somehow* extracting
> out C comment docs and combining w/ sphinx stuff..  maybe there is
> something to re-use from that.

The actual parsing, extraction and generation of reStructureText for
the kernel is handled by scripts/kernel-doc, which is a perl script
based around regex parsing of C code. Jani Nikula has also written a
sphinx extension for using this script which can be seen in
Documentation/sphinx/. As you can probably guess kernel-doc is pretty
linux kernel specific. The script existed before the recent adoption
of sphinx and has support for docbook and man page generation. Being
based on regex parsing it has a somewhat limited understanding of the
underlying types of symbols being documented.

I recently raised the idea of using clang's python bindings to extract
kerneldoc on dri-devel due to some of the limitations of kernel-doc I
was hitting and pointed at an experimental tool I wrote last year for
generating reStructureText based on gtk-doc comments:
https://github.com/rib/clib/blob/master/site/rst-from-c.py Probably
not developed enough to be very useful here, but provides one basic
example of generating reStructuredText from structured comments.

Jani mentioned he had a bit of a side project going to use the same
clang bindings to support extracting doxygen comments, so maybe what
he's been working on could be useful for Mesa too, but I think maybe
that was for doxygen itself so maybe nothing to do with generating
reStructureText.

In the specific case of doxygen format comments, I have a vague
recollection that clang itself can actually parse and provide some
kind of AST for your comments, so maybe that would be convenient.

Br,
- Robert

>
> BR,
> -R
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev