text and compositing

[Jaymz Julian]

On Fri, Apr 23, 2004 at 11:59:15AM -0700, Jon Smirl wrote:
> With xserver apps normally paint into a rectangluar window implemented in
> something like a pbuffer. These windows are then composited onto the display.
> 
> Suppose xserver is transforming my app window into a trapezoid or some other odd
> shape. Does my app paint into the pbuffer and then xserver applies the
> transform, or does xserver apply the transform while my app is painting into the
> pbuffer? The second case would yield a non-rectangluar image which would be
> framed by a transparent area.
> 
> This makes a different in font generation. In the first case the app puts the
> bitmap of the font into the pbuffer and then the pbuffer is transformed. This
> distorts the font. In the second case the font could be generated with the
> correct aliasing for the transformed buffer.

Making this work isn't impossible, but it does require some stupid code 
(I had o add siomne similar functionality to something non-x a while
ago, without breaking other people's apps, heh).

-> Draw Text
    -> was the last op drawing text using the same text renderer?
       -> yes -> add to the previous layer
       -> no  -> if previous layer is an X11 pixmap, reduce to minimum size
          -> create new layer containing this text
          -> create new RGBA layer above that of window size

-> GetImage
   -> call imageFlatten()
   -> return flat pixels

-> PolyWhatever
    -> draw on the top layer

-> PutImage
     -> drawon the top layer


etc, etc.

I'll spare you the rest of the op explanations, but the flatten before
presentation to legacy apps is the important one, and of course easily
hardware accelratable (it's just a composite) for standard RGB windows.  (RGBA
X11 windows are more complicated, see later).  The other advantage, of course,
is that you can then expose the layer functionality to apps that know about
it, and have them be able to do useful thigns like undraw text, or find out
what the text that they drew ages ago is.

There is a bunch of gotchas in the implentation, of course, mostly working
around apps doing stupid things, but the worst that happens is that app loses
re-scalable fonts (an example of that is if an app called GetImage on the
whole window, drew some things on the buffer, and then overdrew the entire
window, our text included, with PutImage, which many apps do.  But there is
very little we cna do about them, otehr than having as many carrots as
possible :-)

This is not without problems, however, which are not caused by stupid apps.

RGB apps don't know that the layer might be RGBA.  But it's easy enough to
filter on input, so all colours become colour|0xff00000

Overdrawaing is one of the more annoying gotchas - apps draw lots of text, and
then get rid of it by drawing stuff over the text so it's not there anymore.
Except that in a layered world it is, it's just no longer visible.  But, since
legacy apps can't manipulate the layers anyhow, it's simple enough to have a
thread which reaps invisible layers, and deletes them

Overdrawing also cause havok with RGBA apps don't work, because if you
overdraw an old piece of text with #0000000, well, it won't behave the same
way (in a traditional app, it will remove the text, in the new app, it will be
transparent, so you will still see the text).  The solution that I used to
this, is to have a special magic colour that apps aren't allowed to draw
(again, filtered at the socket.  it was #11223344 in my case, but it could be
any colour), which represents "No pixel here".  Layers so marked will NOT be
combined with other layers, but instead it will copy every pixel EXCEPT the
ones containing the special colour value.  However, font layers will still be
RGBA, of course, for all of your antialiasing goodness.

This *should* be acceleratable with a texture program on new cards, however I
havn't actually implented this (the library where i added this had no need for
them, and the new thing I'm working on hasn't gotten as far as having this
piece of  code written for hardware yet), so I can't say for certain.  It was
quite fast in software, though, so I don't see why it wouldn't be incredibly
fast in hardware.

There is still a couple of unsolved problems with this plan, the biggest one
being related to AA lines and accelratetion - specially, you have to AA
against the *flattened* image in this case when dealing with an RGBA app that
has no knowladge of layers, which will cause havok when you re-render the
fonts to a different size, changing the pixels.  A possible solution to this
is to render ALL antialiased ops to their own drawable type in this case, but
that could cause massive performance problems if not handled right.  But would
have the advantage that those entities are also smoothly scaled, which is
nothing to complain about.

        -- jj