[RFC wayland] doc: generate doxygen html output from the scanner

Mon Nov 9 20:12:48 PST 2015

On Mon, Nov 09, 2015 at 09:32:52AM +0200, Pekka Paalanen wrote:
> On Sat, 7 Nov 2015 09:53:10 +1000
> Peter Hutterer <peter.hutterer at who-t.net> wrote:
> 
> > On 7/11/2015 01:27 , Pekka Paalanen wrote:
> > > On Thu, 29 Oct 2015 11:48:01 +1000
> > > Peter Hutterer <peter.hutterer at who-t.net> wrote:
> > >
> > >> This switches the scanner to generate doxygen-compatible tags for the
> > >> generated protocol headers, and hooks up the doxygen build to generate server
> > >> and client-side API documentation.
> > >>
> > >> For the wayland protocol, this generates a mainpage with the copyright
> > >> information, all interfaces are separated by doxygen groups and thus listed in
> > >> "Modules" in the generated output.
> > >>
> > >> Function, struct and #defines are documented in doxygen style and associated
> > >> with the matching interface.
> > >> ---
> > >> This is an RFC for now, we need to agree on whether we want to switch to
> > >> doxygen style first. Other changes still missing here are:
> > >> * afaict, the summary can be dropped for most entries, it doesn't seem
> > >>    to add any value if a long description is there
> > >> * currently parsing too many header files, we should only parse the protocol
> > >>    ones for a cleaner documentation.
> > >> * future work for wayland-protocols is to add the various protocols at the
> > >>    @page level
> > >> * a couple of things don't have the doxygen tag yet (mostly #defines)
> > >>
> > >>   doc/doxygen/Makefile.am        |  22 +++++-
> > >>   doc/doxygen/wayland.doxygen.in |   4 +-
> > >>   src/scanner.c                  | 165 ++++++++++++++++++++++++++---------------
> > >>   3 files changed, 130 insertions(+), 61 deletions(-)
> > >
> > > Hi Peter,
> > >
> > > this is a cool idea. I still haven't grasped all the details, but I
> > > think we could use the following sets of "appendices" to the Wayland
> > > documentation (prose):
> > >
> > > A. language-agnostic protocols docs
> > > B. libwayland library API docs, server and client separated
> > > C. protocol C bindings docs, server and client separated
> > >
> > > A and B are what we already have in some form, and C is what you are
> > > now adding. Excellent!
> > >
> > > A is generated directly from XML, and intended for people working on
> > > other language bindings than our C bindings (when we get part C for
> > > people working in C).
> > >
> > > B is generated through Doxygen scanning non-generated code. It is
> > > useful to keep the C bindings docs out of this, so that people writing
> > > other language bindings have a coherent doc and they don't need to
> > > check whether a function listed is something they are supposed to use
> > > or not (i.e. generated or not).
> > 
> > fwiw, it'd be possible to scan the core libraries as well and put them 
> > in a separate doxygen group/page. That way they'd resolve internally 
> > (e.g. an arg of type wl_resource) but still be separated in the doc.
> 
> Hi Peter,
> 
> sounds good.
> 
> > > A and C contain basically the same information, just laid out
> > > differently: C talks in the proper C function and type names, while A
> > > uses just the protocol names. C is further duplicated between server
> > > and client docs.
> > >
> > > I'm not sure if we need server:B+C and client:B+C docs as such, since I
> > > don't think there are many references between B and C. It might be
> > > enough to have one TOC page for "server" and one for "client" which
> > > then somehow link to B and C parts.
> > >
> > > Anyway, that's just random thoughts.
> > >
> > > What I would really like is a way to have links to A, B (and C) from the
> > > *prose*. Looks like we can already have links to A at least.
> > >
> > > What bothers me in the current B from a user point of view is that I
> > > don't know how to get a link to a specific function's documentation. I
> > > don't think there is a TOC listing all functions, and when I find the
> > > function I'm looking for, I can't take a link to it for e.g. pasting in
> > > IRC. Doxygen docs themselves have all the links and TOCs, but the
> > > conversion to docbook loses too much currently.
> > 
> > I think this should be possible to extract with a bit of xsl tweaking, 
> > I'll try to look into that.
> > 
> > One problem with doxygen though is that you can't link directly to a 
> > function from external sources, the anchors are hashes rather than 
> > function names. You can only link to a page, a group and a struct.
> 
> Bummer. I was hoping that there might be some way to create those links
> when passing everything including the prose through a single tool at
> once, but I suppose that's not really possible if there is no output
> mode in Doxygen that didn't use hashes as the anchors. And if the
> hashes are too unstable to use - I think I recall seeing them change
> for no good reason.

apparently they are an md5sum of the function signature and stay the same as
long as the function signature remains constant. So you can link to them
directly, it's just going to be an ugly-looking link.

The GENERATE_TAGFILE option generates a list of all items in xml format like
this:
    <member kind="function">
      <type>struct wl_event_queue *</type>
      <name>wl_display_create_queue</name>
      <anchorfile>wayland-client-core_8h.html</anchorfile>
      <anchor>a6607ab92946184c1ecefba21987b0a83</anchor>
      <arglist>(struct wl_display *display)</arglist>
    </member>

and lo and behold this is exactly the function: 
http://people.freedesktop.org/~whot/wayland-doxygen/wayland/Client/wayland-client-core_8h.html#a6607ab92946184c1ecefba21987b0a83

So the question now is mostly: what exactly do we want to do with this
information? :)

> Unless maybe if we did convert all docbook to doxygen... but I don't
> know what downsides that would have.

Bryce pointed this out in the thread somewhere - you'd break any links to the
current documentation. Which in the case of the client/server API is IMO
low-impact, but for the rest, i.e. protocol and prose, I think docbook is
better.

Cheers,
   Peter

> > > Ah, I hope I gave some ideas. Interesting work!
> > >
> > > When you say "whether we want to switch to doxygen style", do you mean
> > > that we'd drop the conversion to docbook? Would we be working towards
> > > completely replacing docbook with only doxygen?
> > 
> > No, the current docbook is untouched. The difference is simply that we 
> > have a different doc style in the scanner sources that doesn't work for 
> > doxygen (e.g. @foo instead of @param foo). AFAICT we're not using that 
> > style for anything, so switching to doxygen won't hurt us there.
> 
> Sure. I wonder if the old scanner generated comment format follows
> kerneldoc instead, maybe it was just a thinko. I had always assumed it
> was already doxygen format, just unused by us.
> 
> That conversion part is definitely Acked-by me.
> 
> > And since docbook is generated from doxygen, any additional info we 
> > generate that's doxygen compatible will benefit the docbook conversion.
> 
> Cool.
> 
> > > I applied this patch and tried to build it, but it seems it generates
> > > object destructor functions twice, e.g. wl_buffer_destroy, so the build
> > > fails.
> > 
> > Weird. Then again, this was just an RFC patch :) Have a look at the new 
> > patch I sent out, that one works properly and generates nicer output in 
> > general
> > 
> > http://lists.freedesktop.org/archives/wayland-devel/2015-November/025238.html
> 
> I'll try. :-)
> 
> 
> Thanks,
> pq