X.Org documentation formats, plans, conversions, etc.

[Alan Coopersmith]

I've been asked on IRC about the documentation grand plan a few times
lately, and promised to write something up, so here goes, along with
some discussion at the end about possible changes.

This is already somewhat documented in the wiki at:
  http://www.x.org/wiki/Development/Documentation/WritingDocumentation
but there's also some amount of oral history & tradition here that hasn't
been put in the wiki.

In the monolith, documentation was mostly under xc/doc, except for man
pages which were in the directories with the sources of the programs
they documented.

When we split into modules, most of the docs under xc/doc ended up in
doc/xorg-docs.   For most of the docs, this was intended to be only
temporary - we really didn't plan on still having so many there 4 years
later.   The original plan was "Move them to the module they document
when you convert to a format with with widely available, open source
tools for converting to end-user friendly formats like pdf & html."

At the time, that format was generally assumed to be DocBook/XML for
everything.   A list of docs that people claimed to be working on
converting was posted at
   http://www.x.org/wiki/Development/Documentation/DocBookConversion
but few docs were converted.

A while ago we accepted that Plain Text (UTF-8 encoded) was also acceptable,
and the docs for protocols like XFixes & XRandr in those formats moved to
their respective proto modules.

As I was working on the docs for the 7.5 release, I got annoyed at how
little progress we'd made with some of these, and made the unilateral
decision that troff also counted as an acceptable format, especially
given the wide availability of groff.   Unfortunately, groff support across
the distros & OS'es wasn't quite as uniform as I expected and there's
some issues here that need to be cleaned up.   I'd still like to get them
converted to a better format, but they're more useful if they're in the
modules they document, so they're easier to find (I still find docs for
things I didn't realize we had documentation for), easier to keep in
sync with the code, and easier for packagers to include in their packages.

I plan to continue working on moving docs to the modules they document, with
the hope that the 7.6 release version of xorg-docs will have only docs that
really truly don't have a better home (the misc man pages for instance), but
that's a long-term plan, not something that will be done right away.  (Others
should feel free to help at any point - I'm happy to share the workload.)

At this point, these are the documentation input and output formats I know of:

Preferred output formats:
  - man pages for commands and public API in client-side libraries,
	to be installed in $(mandir)
  - plain text, html & pdf for specifications and other documents,
	to be installed in $(docdir) & published on the website

Preferred input formats:
  - DocBook/XML (can generate man, text, html & pdf)

Acceptable input formats:
  - AsciiDoc (can generate DocBook, thus all DocBook supports)
  - Plain text
  - man pages in troff -man format
  - doxygen comments in source code

Deprecated input formats (no new docs should be added in these):
  - troff -ms/-mm
  - FrameMaker
  - LinuxDoc
  - DocBook/SGML
  - TeX

Those in the last list are the ones we most especially want converted to
new formats.   While many of them have open source toolchains available,
we'd like to reduce the number of toolchains we need packagers & builders
to install, and the number of formats developers need to know to be able
to keep our docs up to date.  Generating all the different docs for the
7.5 release website was a lot more painful than it should be, which is
probably why it was skipped for several releases.

We should also settle on a toolchain - I'm assuming that for DocBook/XML
xmlto is preferred over the various docbook2* tools, as it seems to be
more actively maintained.

Peter has also made some arguments on IRC towards changing our official
preferred format from DocBook to AsciiDoc, since the outputs are the same,
but the creation is easier for developers and the raw docs in the source
trees/git repos are easier to read.   I haven't worked with it yet, so
can't comment on my experiences, though the arguments seem reasonable.
Anyone else want to comment/vote?

-- 
	-Alan Coopersmith-           alan.coopersmith at sun.com
	 Sun Microsystems, Inc. - X Window System Engineering