[PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images
Daniel Vetter
daniel.vetter at ffwll.ch
Tue Feb 28 17:13:15 UTC 2017
From: Markus Heiser <markus.heiser at darmarit.de>
This patch brings scalable figure, image handling and a concept to
embed *render* markups:
* DOT (http://www.graphviz.org)
* SVG
For image handling use the 'image' replacement::
.. kernel-image:: svg_image.svg
:alt: simple SVG image
For figure handling use the 'figure' replacement::
.. kernel-figure:: svg_image.svg
:alt: simple SVG image
SVG image example
Embed *render* markups (or languages) like Graphviz's **DOT** is
provided by the *render* directive.::
.. kernel-render:: DOT
:alt: foobar digraph
:caption: Embedded **DOT** (Graphviz) code.
digraph foo {
"bar" -> "baz";
}
The *render* directive is a concept to integrate *render* markups and
languages, yet supported markups:
* DOT: render embedded Graphviz's **DOT**
* SVG: render embedded Scalable Vector Graphics (**SVG**)
v2: s/DOC/DOT/ in a few places (by Daniel).
v3: Simplify stuff a bit (by Daniel):
- Remove path detection and setup/check code for that. In
Documentation/media/Makefile we already simply use these tools,
better to have one consolidated check if we want/need one. Also
remove the convertsvg support, we require ImageMagick's convert
already in the doc build, no need for a 2nd fallback.
- Use sphinx for depency tracking, remove hand-rolled version.
- Forward stderr from dot and convert, otherwise debugging issues with
the diagrams is impossible.
Cc: Jonathan Corbet <corbet at lwn.net>
Cc: linux-doc at vger.kernel.org
Cc: Jani Nikula <jani.nikula at linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab at s-opensource.com>
Signed-off-by: Markus Heiser <markus.heiser at darmarit.de> (v1)
Signed-off-by: Daniel Vetter <daniel.vetter at intel.com>
---
Documentation/conf.py | 2 +-
Documentation/doc-guide/hello.dot | 3 +
Documentation/doc-guide/sphinx.rst | 90 ++++++-
Documentation/doc-guide/svg_image.svg | 10 +
Documentation/process/changes.rst | 7 +-
Documentation/sphinx/kfigure.py | 442 ++++++++++++++++++++++++++++++++++
6 files changed, 548 insertions(+), 6 deletions(-)
create mode 100644 Documentation/doc-guide/hello.dot
create mode 100644 Documentation/doc-guide/svg_image.svg
create mode 100644 Documentation/sphinx/kfigure.py
diff --git a/Documentation/conf.py b/Documentation/conf.py
index f6823cf01275..e3f537ce2935 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ from load_config import loadConfig
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure']
# The name of the math extension changed on Sphinx 1.4
if major == 1 and minor > 3:
diff --git a/Documentation/doc-guide/hello.dot b/Documentation/doc-guide/hello.dot
new file mode 100644
index 000000000000..504621dfc595
--- /dev/null
+++ b/Documentation/doc-guide/hello.dot
@@ -0,0 +1,3 @@
+graph G {
+ Hello -- World
+}
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index 532d65b70500..b902744ce7dd 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -34,8 +34,10 @@ format-specific subdirectories under ``Documentation/output``.
To generate documentation, Sphinx (``sphinx-build``) must obviously be
installed. For prettier HTML output, the Read the Docs Sphinx theme
-(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
-needed. All of these are widely available and packaged in distributions.
+(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
+``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
+from ImageMagick (https://www.imagemagick.org). All of these are widely
+available and packaged in distributions.
To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
@@ -232,3 +234,87 @@ Rendered as:
* .. _`last row`:
- column 3
+
+
+Figures & Images
+================
+
+If you want to add an image, you should use the ``kernel-figure`` and
+``kernel-image`` directives. E.g. to insert a figure with a scalable
+image format use SVG::
+
+ .. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+.. kernel-figure:: svg_image.svg
+ :alt: simple SVG image
+
+ SVG image example
+
+The kernel figure (and image) directive support **DOT** formated files, see
+
+* DOT: http://graphviz.org/pdf/dotguide.pdf
+* Graphviz: http://www.graphviz.org/content/dot-language
+
+A simple example::
+
+ .. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+.. kernel-figure:: hello.dot
+ :alt: hello world
+
+ DOT's hello world example
+
+Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the
+``kernel-render`` directives.::
+
+ .. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+How this will be rendered depends on the installed tools. If Graphviz is
+installed, you will see an vector image. If not the raw markup is inserted as
+*literal-block*.
+
+.. kernel-render:: DOT
+ :alt: foobar digraph
+ :caption: Embedded **DOT** (Graphviz) code.
+
+ digraph foo {
+ "bar" -> "baz";
+ }
+
+The *render* directive has all the options known from the *figure* directive,
+plus option ``caption``. If ``caption`` has a value, a *figure* node is
+inserted. If not, a *image* node is inserted.
+
+Embedded **SVG**::
+
+ .. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg" version="1.1" ...>
+ ...
+ </svg>
+
+.. kernel-render:: SVG
+ :caption: Embedded **SVG** markup.
+ :alt: so-nw-arrow
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full" width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+ </svg>
diff --git a/Documentation/doc-guide/svg_image.svg b/Documentation/doc-guide/svg_image.svg
new file mode 100644
index 000000000000..5405f85b8137
--- /dev/null
+++ b/Documentation/doc-guide/svg_image.svg
@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- originate: https://commons.wikimedia.org/wiki/File:Variable_Resistor.svg -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ version="1.1" baseProfile="full"
+ width="70px" height="40px" viewBox="0 0 700 400">
+ <line x1="0" y1="200" x2="700" y2="200" stroke="black" stroke-width="20px"/>
+ <rect x="100" y="100" width="500" height="200" fill="white" stroke="black" stroke-width="20px"/>
+ <line x1="180" y1="370" x2="500" y2="50" stroke="black" stroke-width="15px"/>
+ <polygon points="585 0 525 25 585 50" transform="rotate(135 525 25)"/>
+</svg>
diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 56ce66114665..e4f25038ef65 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -318,9 +318,10 @@ PDF outputs, it is recommended to use version 1.4.6.
.. note::
Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX``
- version 3.14159265. Depending on the distribution, you may also need
- to install a series of ``texlive`` packages that provide the minimal
- set of functionalities required for ``XeLaTex`` to work.
+ version 3.14159265. Depending on the distribution, you may also need to
+ install a series of ``texlive`` packages that provide the minimal set of
+ functionalities required for ``XeLaTex`` to work. For PDF output you'll also
+ need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).
Other tools
-----------
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
new file mode 100644
index 000000000000..8f749d389dae
--- /dev/null
+++ b/Documentation/sphinx/kfigure.py
@@ -0,0 +1,442 @@
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=C0103
+u"""
+ scalable figure and image handling
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ Sphinx extension which implements scalable image handling.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The build for image formats depence on image's source format and output's
+ destination format. This extension implement methods to simplify image
+ handling from the author's POV. Directives like ``kernel-figure`` implement
+ methods *to* always get the best output-format even if some tools are not
+ installed.For more details take a look at ``convert_image(...)`` which is
+ the core of all conversions.
+
+ * ``.. kernel-image``: for image handling / ``.. image::`` replacement
+
+ * ``.. kernel-figure``: for figure handling / ``.. figure::`` replacement
+
+ * ``.. kernel-render``: for render markup / a concept to embed *render*
+ markups (or languages). Supported markups (see ``RENDER_MARKUP_EXT``)
+
+ + ``DOT``: render embedded Graphviz's **DOC**
+ + ``SVG``: render embedded Scalable Vector Graphics (**SVG**)
+ + ... *developable*
+
+ Used tools:
+
+ * ``dot(1)``: Graphviz (http://www.graphviz.org). If Graphviz is not
+ available, the DOT language is inserted as literal-block.
+
+ * SVG to PDF: To generate PDF, you need at least one of this tools:
+
+ - ``convert(1)``: ImageMagick (https://www.imagemagick.org)
+
+ List of customizations:
+
+ * generate PDF from SVG / used by PDF (LaTeX) builder
+
+ * generate SVG (html-builder) and PDF (latex-builder) from DOT files.
+ DOT: see http://www.graphviz.org/content/dot-language
+
+ """
+
+import os
+from os import path
+import subprocess
+from hashlib import sha1
+import sys
+
+from docutils import nodes
+from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives
+from docutils.parsers.rst.directives import images
+
+from sphinx.directives import patches
+
+__version__ = '1.0'
+
+# simple helper
+# -------------
+
+def mkdir(folder, mode=0o775):
+ if not path.isdir(folder):
+ os.makedirs(folder, mode)
+
+# def debug_handle(self, node): # pylint: disable=W0613
+# from linuxdoc.kernel_doc import CONSOLE
+# CONSOLE()
+
+def pass_handle(self, node): # pylint: disable=W0613
+ pass
+
+# setup conversion tools and sphinx extension
+# -------------------------------------------
+
+def setup(app):
+ # image handling
+ app.add_directive("kernel-image", KernelImage)
+ app.add_node(kernel_image,
+ html = (visit_kernel_image, pass_handle),
+ latex = (visit_kernel_image, pass_handle),
+ texinfo = (visit_kernel_image, pass_handle),
+ text = (visit_kernel_image, pass_handle),
+ man = (visit_kernel_image, pass_handle), )
+
+ # figure handling
+ app.add_directive("kernel-figure", KernelFigure)
+ app.add_node(kernel_figure,
+ html = (visit_kernel_figure, pass_handle),
+ latex = (visit_kernel_figure, pass_handle),
+ texinfo = (visit_kernel_figure, pass_handle),
+ text = (visit_kernel_figure, pass_handle),
+ man = (visit_kernel_figure, pass_handle), )
+
+ # render handling
+ app.add_directive('kernel-render', KernelRender)
+ app.add_node(kernel_render,
+ html = (visit_kernel_render, pass_handle),
+ latex = (visit_kernel_render, pass_handle),
+ texinfo = (visit_kernel_render, pass_handle),
+ text = (visit_kernel_render, pass_handle),
+ man = (visit_kernel_render, pass_handle), )
+
+ return dict(
+ version = __version__,
+ parallel_read_safe = True,
+ parallel_write_safe = True
+ )
+
+# integrate conversion tools
+# --------------------------
+
+RENDER_MARKUP_EXT = {
+ # The '.ext' must be handled by convert_image(..) function's *in_ext* input.
+ # <name> : <.ext>
+ 'DOT' : '.dot'
+ , 'SVG' : '.svg'
+}
+
+def convert_image(img_node, translator): # pylint: disable=R0912
+ """Convert an image node for the builder.
+
+ Different builder prefer different image formats, e.g. *latex* builder
+ prefer PDF while *html* builder prefer SVG format for images.
+
+ This function handles outputs image formats in depence of source the format
+ of the image and the translator's output format. This also means to
+ manipulate/update the *image* dictionary of the builder (``builder.images``)
+
+ """
+ fname, in_ext = path.splitext(path.basename(img_node['uri']))
+ src_fname = path.join(translator.builder.srcdir, img_node['uri'])
+ src_folder = path.dirname(img_node['uri'])
+ out_dir = translator.builder.outdir
+ dst_fname = None
+
+ # in kernel builds, use 'make SPHINXOPTS=-v' to see verbose messages
+ verbose = translator.builder.app.verbose
+ warn = translator.builder.warn
+
+ verbose('assert best format for: ' + img_node['uri'])
+
+ if in_ext == '.dot':
+
+ # ----------
+ # handle DOT
+ # ----------
+
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if translator.builder.format == 'html':
+ dst_fname = path.join(out_dir, src_folder, fname + '.svg')
+ else:
+ # all other builder formats will include DOT as raw
+ with open(src_fname, "r") as dot:
+ data = dot.read()
+ node = nodes.literal_block(data, data)
+ img_node.replace_self(node)
+
+
+ elif in_ext == '.svg':
+
+ # ----------
+ # handle SVG
+ # ----------
+
+ if translator.builder.format == 'latex':
+ dst_fname = path.join(out_dir, fname + '.pdf')
+
+ if dst_fname:
+ name = dst_fname[len(out_dir) + 1:]
+ # the builder needs not to copy one more time, so pop it if exists.
+ translator.builder.images.pop(img_node['uri'], None)
+ img_node['uri'] = dst_fname
+ img_node['candidates'] = {'*': dst_fname}
+
+ mkdir(path.dirname(dst_fname))
+
+ if in_ext == '.dot':
+ verbose('convert DOT to: {out}/' + name)
+ dot2format(src_fname, dst_fname)
+
+ elif in_ext == '.svg':
+ verbose('convert SVG to: {out}/' + name)
+ svg2pdf(src_fname, dst_fname)
+
+def dot2format(dot_fname, out_fname):
+ """Converts DOT file to ``out_fname`` using ``dot(1)``.
+
+ * ``dot_fname`` pathname of the input DOT file, including extension ``.dot``
+ * ``out_fname`` pathname of the output file, including format extension
+
+ The *format extension* depends on the ``dot`` command (see ``man dot``
+ option ``-Txxx``). Normally you will use one of the following extensions:
+
+ - ``.ps`` for PostScript,
+ - ``.svg`` or ``svgz`` for Structured Vector Graphics,
+ - ``.fig`` for XFIG graphics and
+ - ``.png`` or ``gif`` for common bitmap graphics.
+
+ """
+ out_format = path.splitext(out_fname)[1][1:]
+ cmd = ['dot', '-T%s' % out_format, dot_fname]
+ exit_code = 42
+ with open(out_fname, "w") as out:
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ out.flush()
+ return bool(exit_code == 0)
+
+def svg2pdf(svg_fname, pdf_fname):
+ """Converts SVG to PDF with CairoSVG or ``convert(1)`` command.
+
+ Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
+ conversion. Returns ``True`` on success and ``False`` if an error occurred
+ (e.g. none of the conversion tool is available).
+
+ * ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
+ * ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
+
+ """
+ cmd = [convert_cmd, svg_fname, pdf_fname]
+ p = subprocess.Popen(
+ cmd, stdout = out, stderr = subprocess.PIPE )
+ nil, err = p.communicate()
+
+ sys.stderr.write(err)
+
+ exit_code = p.returncode
+ return bool(exit_code == 0)
+
+
+# image handling
+# ---------------------
+
+def visit_kernel_image(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_image`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0]
+ convert_image(img_node, self)
+
+class kernel_image(nodes.General, nodes.Element):
+ """Node for ``kernel-image`` directive."""
+ pass
+
+class KernelImage(images.Image):
+ u"""KernelImage directive
+
+ Earns everything from ``.. image::`` directive, except *remote URI* and
+ *glob* pattern. The KernelImage wraps a image node into a
+ kernel_image node. See ``visit_kernel_image``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s": glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = images.Image.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (image_node,) = result
+ # wrap image node into a kernel_image node / see visitors
+ node = kernel_image('', image_node)
+ return [node]
+
+# figure handling
+# ---------------------
+
+def visit_kernel_figure(self, node): # pylint: disable=W0613
+ """Visitor of the ``kernel_figure`` Node.
+
+ Handles the ``image`` child-node with the ``convert_image(...)``.
+ """
+ img_node = node[0][0]
+ convert_image(img_node, self)
+
+class kernel_figure(nodes.General, nodes.Element):
+ """Node for ``kernel-figure`` directive."""
+
+class KernelFigure(patches.Figure):
+ u"""KernelImage directive
+
+ Earns everything from ``.. figure::`` directive, except *remote URI* and
+ *glob* pattern. The KernelFigure wraps a figure node into a kernel_figure
+ node. See ``visit_kernel_figure``.
+ """
+
+ def run(self):
+ env = self.state.document.settings.env
+
+ uri = self.arguments[0]
+ if uri.endswith('.*') or uri.find('://') != -1:
+ raise self.severe(
+ 'Error in "%s: %s":'
+ ' glob pattern and remote images are not allowed'
+ % (self.name, uri))
+
+ # Tell sphinx of the dependency
+ env.note_dependency(os.path.abspath(uri))
+
+ result = patches.Figure.run(self)
+ if len(result) == 2 or isinstance(result[0], nodes.system_message):
+ return result
+ (figure_node,) = result
+ # wrap figure node into a kernel_figure node / see visitors
+ node = kernel_figure('', figure_node)
+ return [node]
+
+
+# render handling
+# ---------------------
+
+def visit_kernel_render(self, node):
+ """Visitor of the ``kernel_render`` Node.
+
+ If rendering tools available, save the markup of the ``literal_block`` child
+ node into a file and replace the ``literal_block`` node with a new created
+ ``image`` node, pointing to the saved markup file. Afterwards, handle the
+ image child-node with the ``convert_image(...)``.
+ """
+
+ verbose = self.builder.app.verbose
+ warn = self.builder.warn
+ srclang = node.get('srclang')
+
+ verbose('visit kernel-render node lang: "%s"' % (srclang))
+
+ tmp_ext = RENDER_MARKUP_EXT.get(srclang, None)
+ if tmp_ext is None:
+ warn('kernel-render: "%s" unknow / include raw.' % (srclang))
+ return
+
+ literal_block = node[0]
+ code = literal_block.astext()
+
+ if tmp_ext:
+ hashobj = code.encode('utf-8') # str(node.attributes)
+ fname = '%s-%s' % (srclang, sha1(hashobj).hexdigest())
+ tmp_fname = path.join(
+ self.builder.outdir, self.builder.imagedir, fname + tmp_ext)
+
+ if not path.isfile(tmp_fname):
+ mkdir(path.dirname(tmp_fname))
+ with open(tmp_fname, "w") as out:
+ out.write(code)
+
+ image_node = nodes.image(node.rawsource, **node.attributes)
+ image_node['uri'] = tmp_fname
+
+ literal_block.replace_self(image_node)
+ convert_image(image_node, self)
+
+
+class kernel_render(nodes.General, nodes.Inline, nodes.Element):
+ """Node for ``kernel-render`` directive."""
+ pass
+
+class KernelRender(patches.Figure):
+ u"""KernelRender directive
+
+ Render content by external tool. Has all the options known from the
+ *figure* directive, plus option ``caption``. If ``caption`` has a
+ value, a figure node with the *caption* is inserted. If not, a image node is
+ inserted.
+
+ The KernelRender directive wraps the text of the directive into a
+ literal_block node and wraps it into a kernel_render node. See
+ ``visit_kernel_render``.
+ """
+ has_content = True
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = False
+
+ # earn options from 'figure'
+ option_spec = patches.Figure.option_spec.copy()
+ option_spec['caption'] = directives.unchanged
+
+ def run(self):
+ return [self.build_node()]
+
+ def build_node(self):
+
+ srclang = self.arguments[0].strip()
+ if srclang not in RENDER_MARKUP_EXT.keys():
+ return [self.state_machine.reporter.warning(
+ 'Unknow source language "%s", use one of: %s.' % (
+ srclang, ",".join(RENDER_MARKUP_EXT.keys())),
+ line=self.lineno)]
+
+ code = '\n'.join(self.content)
+ if not code.strip():
+ return [self.state_machine.reporter.warning(
+ 'Ignoring "%s" directive without content.' % (
+ self.name),
+ line=self.lineno)]
+
+ node = kernel_render()
+ node['alt'] = self.options.get('alt','')
+ node['srclang'] = srclang
+ literal_node = nodes.literal_block(code, code)
+ node += literal_node
+
+ caption = self.options.get('caption')
+ if caption:
+ # parse cation's content
+ parsed = nodes.Element()
+ self.state.nested_parse(
+ ViewList([caption], source=''), self.content_offset, parsed)
+ caption_node = nodes.caption(
+ parsed[0].rawsource, '', *parsed[0].children)
+ caption_node.source = parsed[0].source
+ caption_node.line = parsed[0].line
+
+ figure_node = nodes.figure('', node)
+ for k,v in self.options.items():
+ figure_node[k] = v
+ figure_node += caption_node
+
+ node = figure_node
+
+ return node
+
--
2.11.0
More information about the dri-devel
mailing list