[gst-devel] Text rendering

[Gergely Nagy]

On Sun, Feb 20, 2005 at 12:01:43AM +0100, Maciej Katafiasz wrote:
> > Anyway, for fancy effects (think not only in colors, but textures too),
> > the approach outlined here will work fine, I guess.
> 
> In a word, no. It's not gonna work for subtitles at all (even simple
> colouring of different lines in different colours will be difficult, and
> individually coloured letters will be downright impossible).

For this, I imagined that the text description would include color markup
and such. The text renderer would ignore that, and there would be another
element, which would set up the coloring layer.

Something like:

                                    +---------------+
+-------------+ application/        | Text renderer |
| Sub. parser |------------------+--+---------------+---+-------+
+-------------+ x-gst-subtitles  |                      | Image |
                                 |                      | mixer |
                                 +--+---------------+---+---+---+
                                    | Color-layer   |       |
                                    | renderer      |       |
                                    +---------------+       |
                                                           /
                                                          /
+--------------+                       +-----------------+
| Video source |-----------------------+ Image mixer     |
+--------------+                       +-------+---------+
                                               |
                                       +-----------------+
                                       | Video out       |
                                       +-----------------+

This means a bit moe elements, but these elements would be simple. I like
simple elements, that do only one tiny little thing.

However, since calculating the size of fonts is not easy, and that info
is required for properly creating a layer of only color, text-renderer
might also emit something like a `bounding-box markup'..

On the other hand, in the end, we end up writing a rendering system, built
upon small components. Doing all this in one element with cairo from the
start might be far better to start with.

> That's because whilst presented model is rather flexible when it works,
> it's also extremely static (you'd need to replug pipeline *each* time you
> wanted to have differently coloured line added, yuck!), and makes
> complex things even more complex, losing lots of info in the process.

Don't think so... just fiddle some properties of the coloring element,
and you're fine.

> What I want to see is *one* (in practice that's all that should be
> needed, but we can make it subclassable for really special needs)
> element supporting range of basic, but rich, and most importantly,
> combinable ops from which we can build effects. IOW, we want (cairo)
> canvas, and a protocol for manipulating its objects, which various
> parsers and other elements that want to generate text will use.
> 
> Mandatory ASCII-art:
> 
> +----------+                 +-----------+            +-----------+
> |          | application/    |   Text    |  video/    |           |
> | Subtitle +-----------------+ Renderer  +------------+   Image   |
> |  Parser  | x-gst-subtitles | (cairo    |  x-raw-rgb |   mixer   |
> |          |                 | overlay)  |  (ARGB)    |           |
> +----------+                 +-----------+            +-----------+
>                                                           /
>                                                          /
>                                                         /
>                                                        /
> +----------+                                          /
> |          |                                         /
> |  Video   |     video/x-raw-rgb                    /
> | renderer +---------------------------------------/
> |          |
> +----------+
> 
> 
> It outputs into imagemixer instead of blitting directly to video because
> Gergely wants to have output in non-blitted way to have fun with his toy
> elements afterwards :). This way we can keep most of his proposal, that
> is apply separate *video* effects, whilst having robust *text*
> rendering.

Sounds good to me so far.

> Now, application/x-gst-subtitles is a protocol that would support:
> 
> - creating objects with unique ID
> - manipulating (move, resize, rotate, colourise, maybe arbitrary
> transformation) objects with given ID
> - rendering (it should be operation separate from creation) objects
> - destroying objects

Now, this is something I don't completely agree with. This sounds like
subtitle rendering would be performed onto a canvas that has the size
of the video, while the subtitles themselves might only be a small
fraction of the whole thing. Now, blending a 720x576 image onto another
is much more costy than blending a 720x48 image onto a 720x576 at
position (0,650) (for example).

Some moving might have a place in the x-gst-subtitles protocol, but..
support for scrolling should not be there, imho. What has its place
there in my opinion, is line alignment or the like..

Hrm.. I think I'll think a bit more about this, and send another
reply again, later today. I hope to be able to have some nice
pipeline ideas by then, to illustrate what I have in mind.

-- 
Gergely Nagy