[Mesa-dev] [Request for Comments] - Port documentation to Markdown

[Brian Paul]

On 03/07/2017 02:47 PM, Jason Ekstrand wrote:
> Hey Jean,
>
> On Tue, Mar 7, 2017 at 4:02 AM, Jean Hertel <jean.hertel at hotmail.com
> <mailto:jean.hertel at hotmail.com>> wrote:
>
>     Eric,
>
>      >>On Monday, 2017-03-06 18:36:32 +0000, Jean Hertel wrote:
>      >> Hello guys,
>      >>
>      >>
>      >> I want to propose a port of the current HTML documentation to a
>     markdown-like syntax.
>      >>
>      >
>      >I was thinking of doing essentially the same thing, just haven't
>     had the
>      >time to do so yet. Please CC me (eric at engestrom.ch
>     <mailto:eric at engestrom.ch>) on anything related
>      >to this project :)
>      >
>      >>
>      >> Current problems
>      > >
>      > >
>      > > Right now I see some problems with the approach used.
>      > >
>      > >   *   People maintaining the documentation must know basic HTML
>     syntax;
>      > >   *   There is no easy way to add a new release page;
>      > >      *   You need to open the index file and make changes there;
>      > >      *   You need to make a new file that will say what has
>     been changed;
>      > >      *   You need to remember to update the release notes pages;
>      > >   *   The current website has many HTML tags not being closed;
>      >>   *   The website don't fit well in mobile area. (The layout is
>     not responsive);
>      >>   *   Making any change to the layout is very hard, because you
>     need to rewrite many pages;
>      >>   *   The current website doesn't show all the possibilities of
>     the library. The website has a 1990 layout, and I think having a new
>     layout would benefit the project;
>      >
>      >Agreed on all counts :)
>      >
>      >>
>      >> Proposal for Markdown
>      > >
>      > > Rewriting the documentation in a markdown syntax can bring some
>     benefits to the development, like:
>      > >
>      > >   *   Anyone writing for freedesktop.org
>     <http://freedesktop.org>, StackOverflow and Github already knows the
>     markdown syntax.
>      > >   *   Independence of HTML. The developer don't need to worry
>     if the HTML output will be correct. The generator makes all the
>     translation from markdown to HTML;
>      >
>      > Agreed, but more on this below.
>
>      > >   *   Easily change the layout of the site, simply changing the
>     theme; (I don't think you change the layout much, but allowing it to
>     be easy wont hurt)
>      >
>      > Having a proper (generated?) HTML structure and a clean CSS is
>     all you
>      > need for this.
>      >
>     I agree.
>
>     > >   *   Creating a new release post can be maded easy, using some static site generator, for example, pelican;
>     >
>     > This can be done with any old tool. You could even have a simple
>     > release.template and copy it and modify it as you need to make a new
>     > release.
>
>     > >
>     > >
>     > > Right now I have maded a partial port from HTML to Markdown.
>     > >
>     >> The code can be found here:https://github.com/jlHertel/mesa-pelican
>     <https://github.com/jlHertel/mesa-pelican>
>     >
>     >I looked at your conversion on a couple random files, and it looks
>     >pretty good to me. (I have a couple nit-picks obviously :P, but the
>     >basic concept is fine.)
>
>     If you find any problem, you can report them directly via a github
>     issue or send me an email. As soon as possible I will fix it.
>
>     >>
>     >> I'm using the pelican static site generator with a layout ported from Wordpress using Twitter Bootstrap CSS.
>     >>
>     >> A live preview can be found here:http://mesa.jeanhertel.com.br/
>
>
> Ok, that looks amazing...
>
>     >> The static site generator tool and the theme used can be ignored if you want, the point here is to rewrite the documentation in a way that is easy to maintain and that >>don't need a good knowledge in HTML.
>     >
>     >I haven't look into Pelican (and won't have time to do so before the
>     >weekend), but I think it sounds like a reasonable kind of requirement to
>     >have. There are plenty of other markdown-to-html generators around, if
>     >someone prefers another one.
>
>     Agree, any generator is fine for me.
>
>     >One thing that I would prefer so not see if heavy things like Bootstrap.
>     >We definitely don't need it, I think writing our own few lines of CSS
>     >(which can be inspired by anything you want) is better. We have more
>     >than enough people who know how to do it (myself included), it will be
>     >cleaner (we won't need to include the whole forest to get our tree) and
>     >much easier to fix when there's a bug.
>
>
> I would tend to agree but I don't care too much about those details so
> long as it's maintainable.  My primary concern is that while a lot of
> random developers in the community are liable to have brushed into CSS a
> time or two, most probably won't know bootstrap.

Yeah, I can's stress that too much.  The site has to be easily 
maintainable by the developers.  I, for one, don't know much about 
websites beyond html and a little CSS.  If you create a new website 
infrastructure and then disappear after a few months we need to be able 
to take over.  Also, we can't funnel documentation updates through a 
handful of people that know a complex system.


>
>     As I already answered Emil, I have put it there because of mylack of
>     knowledge on doing good CSS.
>     Of course you're welcome to help me write a more clean CSS. Any help
>     and suggestion is very apreciated.
>
>     >One core thing that might need to be changed is Markdown itself. It is
>     >great for prose, but it breaks down quite quickly when you start to
>     >write code in it. For this reason, people are moving to RST instead for
>     >code documentation.
>     >
>     >The question becomes: how much code do we expect to put on the website,
>     >and how complex will it be?
>     >
>     >I had in mind that we could start adding small code examples to help new
>     >learners to understand how the apis are used. No tutorial-style articles,
>     >just enough to understand that "you need to init this, then query that,
>     >and pass arguments like this, and handle errors like that".
>
>     I have never heard of problems writing code inside Markdown. My own
>     blog has some code and uses Markdown without problems.
>     Can you please give me a link explaining what are the common problems?
>     Maybe we should worry with this before doing anything.
>
>
> How much are we trying to just make the website easier to update vs. how
> much are we trying to expand documentation?  If we're just trying to
> give the website a face-lift, then something like pelian is probably
> sufficient.  If we want to start putting substantial pieces of
> documentation into the website, then using a more complete tool such as
> Sphinx may be in order.  More on that below.
>
>     >If we ever need to convert from Markdown to RST, I think the conversion
>     >would be pretty trivial, so baring objections I'd recommend going with
>     >Markdown for now.
>
>     I agree that the conversion seems easy.
>     Of course the choosen tool (pelican) supports both Markdown and Rst.
>     You cannot mix them in the same document, but having some documents
>     in RST and some in Markdown is fine.
>
>
> First off, it's worth throwing out there that this is not the first
> proposal to do something like this, just the proposal with the prettiest
> demo :-)  A few references for you:
>
> https://lists.freedesktop.org/archives/mesa-dev/2016-August/127178.html
> <https://lists.freedesktop.org/archives/mesa-dev/2016-August/127178.html>
> https://lists.freedesktop.org/archives/mesa-dev/2015-June/085786.html
> <https://lists.freedesktop.org/archives/mesa-dev/2015-June/085786.html>
> https://lists.freedesktop.org/archives/mesa-dev/2016-December/138196.html
>
> In particular, the conversion of the entire mesa website to rst has
> already been done... twice.

Yeah, we seem to get a new website proposal every few months.  I 
wouldn't be surprised if someone else comes along in six months with 
another new idea.  I'd like to avoid a lot of churn.

>
> One thing that I would personally like to see is for us to start
> building some Sphinx-based architecture-level documentation for mesa.

Yeah, I'd like to see more documentation for the newer Mesa features 
like OpenCL, VDPAU, Vulkan and NIR.


> There is already some substantial gallium documentation written using
> RST and Sphinx and I have some pretty good chunks of NIR and ISL
> documentation floating around that I'd like to merge at some point but
> there's no good place to put it at the moment.  Whether this sort of
> documentation ties into the website or is its own thing that developers
> just build from the git tree, I don't care too much.  However, given the
> way Sphinx is designed, just doing everything in Sphinx seems like a
> reasonable thing to do.

I don't know the strengths of Sphinx vs. Pelican vs. other options, but 
again, let's keep it simple and maintainable.  I still haven't had time 
to look at Bootstrap, but comments saying that it's "heavy" are a big 
reg flag to me.

We already have some Gallium docs in Sphinx.  It would be nice if we 
auto-generated HTML and hosted it on freedesktop, rather than the link 
to https://gallium.readthedocs.io/en/latest/

-Brian