[CREATE] OpenRaster specification: updates to support masking

[Andrew Chadwick]

I'm in the process of (slowly and experimentally) refactoring the layers
code in MyPaint to add a bunch of fancy features like masking, nested
layers, and layer formats other than raster (but which either rasterize
(like SVG) or can be represented usefully as an icon (like basically
nothing right now)).

I've noticed that the OpenRaster specification will need to be updated
to support the Porter-Duff "in" operator, and I'd like to take the
opportunity to allow sub-stacks to be composited with user-specifiable
blending and compositing operators. Conveniently enough, the W3C
Compositing and Blending Level 1 specification has evolved into a very
helpful and complete form, and defines neatly an important aspect of how
"groups" in formats like SVG - equivalent to our nested stacks - should
be expected to render.

Therefore I'd like to update the OpenRaster draft specification[1] in
accordance with the attached proposal. See
https://gist.github.com/achadwick/7827931 in case the attachment hasn't
made it through the mailing list software.


[1] http://www.freedesktop.org/wiki/Specifications/OpenRaster/Draft

-- 
Andrew Chadwick
-------------- next part --------------

0. Updates to the OpenRaster Stack Specification
================================================

The `OpenRaster stack specification`__ is missing some details which need to be added to make masking possible. It also doesn't specify how sub-stacks should to composite on top of other elements in a stack, although in practice this has not mattered so far.

.. __: http://www.freedesktop.org/wiki/Specifications/OpenRaster/Draft/LayersStack/

I'm intending to add sub-layers and masking to MyPaint, and as OpenRaster is our primary (and only!) layered file format, we need to update the specification. In keeping with previous work on the OpenRaster spec, I'd like to use as much of the W3C work on compositing and blending as possible since we already point to that as the model to use for compositing layers. We currently point to a 2004 W3C Working Draft for `SVG Compositing`_, augmented by an evolving Working Draft addressing `Compositing and Blending`_.

.. _SVG Compositing: http://dev.w3.org/SVG/modules/compositing/master/SVGCompositing.html
.. _Compositing and Blending: http://www.w3.org/TR/2012/WD-compositing-20120816/

The `Compositing and Blending Level 1`_ specification is approaching becoming a Candidate Recommendation for Canvas 2D, SVG, HTML and related web technologies. I would like to use this updated document and the SVG group model it describes as the model for OpenRaster stacking.

.. _`Compositing and Blending Level 1`: http://www.w3.org/TR/compositing-1/


1. Proposed Changes in Detail
=============================

1.0. Separate compositing from blending, add Porter-Duff ops
------------------------------------------------------------

The OpenRaster ``<layer/>`` element currently has a ``composite-op`` attribute which is based on the SVG 1.2 standard, plus a few extra non-separable modes from an early draft of `Compositing and Blending Level 1`_.  These all operate as `blending operations`_ which combine the RGB values from a top layer and a bottom layer into a resultant RGB value. This *blending* result is then *composited* onto the target layer using an implicit `Porter-Duff operator`_ (almost always "_over_" according to the current OpenRaster spec), and it is during the compositing phase that the alpha values of the top and bottom layers are used.

.. _`blending operations`: http://www.w3.org/TR/compositing-1/#generalformula
.. _`Porter-Duff operator`: http://www.w3.org/TR/compositing-1/#porterduffcompositingoperators_srcover

Note that the current OpenRaster draft only permits use of the "_over_" operator at this point. Masking demands use of Porter-Duff "_in_", and it is of note that in the GIMP, layers with modes other than "normal" composite using Porter-Duff "_in_" after the blend result is calculated, whereas in MyPaint "_over_" is (almost) always used. To make the two programs fully interoperable, both need to be explicit about their implicit behaviour.

Adopting more of the `Compositing and Blending Level 1`_ spec would allow us to cover these use cases. Therefore I propose:

* Introducing a ``blend`` attribute to ``<layer/>``, permitting any value from `Compositing and Blending Level 1`_'s ``<blend-mode>`` production:

    :Values: normal | multiply | screen | overlay | darken | lighten | color-dodge | color-burn | hard-light | soft-light | difference | exclusion | hue | saturation | color | luminosity
    :Ref: http://www.w3.org/TR/compositing-1/#ltblendmodegt
    :Default: normal

* Introducing a ``composite`` attribute to ``<layer/>``, permitting any value from `Compositing and Blending Level 1`_'s ``<composite-mode>`` production:

    :Values: clear | copy | destination | source-over | destination-over | source-in | destination-in | source-out | destination-out | source-atop | destination-atop | xor | lighter
    :Ref: http://www.w3.org/TR/compositing-1/#ltcompositegt
    :Default: source-over

* Deprecating the ``composite-op`` attribute in ``<layer/>``. This attribute, if present, is to be interpreted according to the following table:

    ===============  ===========  ===========
    composite-op     blend        composite
    ===============  ===========  ===========
    svg:src-over     normal       source-over
    svg:plus         normal       lighter
    svg:multiply     multiply     source-over
    svg:screen       screen       source-over
    svg:overlay      overlay      source-over
    svg:darken       darken       source-over
    svg:lighten      lighten      source-over
    svg:color-dodge  color-dodge  source-over
    svg:color-burn   color-burn   source-over
    svg:hard-light   hard-light   source-over
    svg:soft-light   soft-light   source-over
    svg:difference   difference   source-over
    svg:color        color        source-over
    svg:luminosity   luminosity   source-over
    svg:hue          hue          source-over
    svg:saturation   saturation   source-over
    ===============  ===========  ===========

  Explicit attribute values for ``blend`` and ``composite`` override the implicit ones coming from ``composite-op``.


1.1. Compositing and Blending of Sub-stacks
-------------------------------------------

At present, the specification for ``<stack/>`` does not describe how sub-stacks are to be composited onto their backdrop. In other words, placing a layer in a sub-stack cannot affect its appearance compared to when it is not placed in a sub-stack. There is no visual difference between

* ``A + B + C``     and
* ``A + (B + C)``   and
* ``(A + B) + C``   ,

if stacks are represented by brackets. This is precisely how MyPaint operates currently, and it is equivalent to the concept of `group invariance`_ as described in `Compositing and Blending Level 1`_.

It would be useful to support opacities for sub-groups, to allow them to be toggled off and on by the user, and to permit them to be composited as sub-stacks proper, with the result of their internal compositing being composited onto their backdrop in a configurable manner.  Therefore I propose:

* Introducing the existing ``opacity`` attribute to ``<stack/>``. Using any value for this attribute other than "1.0" for the stack breaks `group invariance`_, and causes the stack to be an isolated group, as defined below.

* Introducing the existing ``visibility`` attribute to ``<stack/>``. Using any value other than "visible" for the stack breaks `group invariance`_. If the value is ``hidden``, the stack is not rendered.

* Introducing the new ``blend`` and ``composite`` attributes as defined above to the definition of ``<stack/>``. No need for the backward compatibility table. Using any blend mode for the stack other than ``normal`` or any composite operator other than ``source-over`` also breaks group invariance, and causes the stack to be an isolated group.

* Introducing a new ``isolation`` attribute as defined in `Compositing and Blending Level 1`_

  :Values: isolated | auto
  :Default: auto

  where the ``isolated`` value breaks `group invariance`_, and causes the stack to be an isolated group. This allows stacks to be explicitly marked as isolated.  The default value, ``auto``, means that the stack's isolation is determined by examining its other attributes as described above.

These attributes are not permitted on the root ``<stack/>`` element: the root stack is always an isolated group, and always composites with ``normal`` blend mode, and the ``source-over`` composite operator over an application defined backdrop.

Stacks which are *isolated groups* first have their contents composited in turn to a zero-alpha, black backdrop (R, G, B and A all zero) instead of to the stack's own backdrop.  This first compositing is unaffected by the attributes of the stack element itself.  The result of this sequence of compositing operations is then composited onto the stack's own backdrop, using the stack's compositing and positioning attributes.

Stacks which are not isolated groups are blended and composited simply by blending and then compositing each of their layers or sub-stacks in turn to their backdrop. The backdrop of the first element is the stack's own backdrop, and the result of the blend and composite operation is used as the backdrop to the next.

While conceptually simple, this model allows for complexities such as implementing multiple alternative or combinable masks on a "layer" as sub-sub-stacks with multiple layers::

    <stack x="42" y="101" opacity="0.5" name="masked-tint" blend="color">
      <layer name="tint" src="data/image1.png" composite="source-in"/>
      <stack name="masks">
        <layer name="mask3" src="data/mask3.png"/>
        <layer name="mask2" src="data/mask2.png"/>
        <layer name="mask1" src="data/mask1.png" visibility="hidden"/>
      </stack>
      <!-- OpenRaster stacks are applied in the reverse of document order -->
      <!-- Start here -->
    </stack>

which has the advantage that programs with differing user interfaces but support for sub-stack blending and compositing can still manipulate and preserve these masks. The example is rendered as follows:

* The "``masked-tint``" stack is an isolated group because although its isolation is ``auto`` (the default), it has an opacity which is not exactly 1.0. So we render it by first rendering its contents to a blank background

  * either: ``blank + mask2 + mask3 + tint``
  * or: ``blank + (mask2 + mask3) + tint``
  * or: ``((blank + mask2) + mask3) + tint``

  all are equivalent, because the "``tint``" layer is blended with and then composited into its backdrop, and its backdrop consists of the union of the visible layers in "``masks``" and no layers other than the implicit blank layer.

* The results of this sub-rendering are then composited over the backdrop of "``masked-tint``" using the ``color`` blend mode, the ``source-over`` Porter-Duff compositing operator, and an opacity of 0.5.

.. _group invariance: http://www.w3.org/TR/compositing-1/#groupinvariance