[HarfBuzz] HarfBuzz 1.0 API; the message you were hoping would never come

[Jonathan Kew]

On 3/1/14 06:12, James Clark wrote:
> On Mon, Dec 30, 2013 at 7:26 PM, Jonathan Kew <jfkthame at googlemail.com
> <mailto:jfkthame at googlemail.com>> wrote:
>
>
>     There's overlap here with the process of font-matching (choosing the
>     font(s) to be used for a given text sequence), which is clearly out
>     of scope for harfbuzz. If a given Unicode character is not supported
>     (exactly, or via a *canonical* [de]composition) by a given font,
>     there are several possible outcomes: just render the font's .notdef
>     glyph; render some synthetic representation of the codepoint
>     (hexbox); render a compatibility-equivalent character/sequence, if
>     such exists; choose a different font.
>
>
> I agree with this model.  There's a general problem of what you might
> call "fallback": what to do if the requested font does not have a glyph
> for the requested character.
>
> This makes me realize that I don't understand the big picture of how
> this fallback process interacts with harfbuzz. In order to do fallback,
> you need to do character to glyph mapping.

Not necessarily. You need to know the character repertoire supported by 
the font, but you may not need to actually map to glyphs. In Firefox, 
for instance, font fallback is done based on a per-font *bit* map of 
supported Unicode codepoints. So at the font fallback stage, we know 
whether the character is present, but do not map it to a glyph.

> If the application is
> expected to do fallback before calling harfbuzz, why does harfbuzz
> expect chars rather than glyphs as input to the shaping process?

One good reason, at least, would be that shaping requires harfbuzz to 
have access to Unicode character properties, and it is not necessarily 
possible to derive these from glyph IDs.

> I
> would have expected there to be some application callback that harfbuzz
> would call when there is a need to do fallback; the application would
> use this to tell harfbuzz how to handle this situation for this
> particular character. Some of these harfbuzz could handle by itself,
> others would require cooperation between the application and harfbuzz.

This doesn't fit into the harfbuzz model, where the "unit of work" is a 
buffer that is shaped with one specified font according to the rules of 
one script and language system.

Harfbuzz does not attempt to manage larger blocks of text that may 
involve multiple directions, scripts, languages or fonts. All this 
belongs (IMO) to a higher level of the software stack (such as Pango), 
which will require a richer data model than HB's buffer, and is 
responsible for font fallback as well as for dividing the text into 
direction/script/language/font runs, each of which can be separately 
shaped by harfbuzz.

Using compatibility decompositions to provide a fallback rendering would 
be one of the options such a higher-level component might use. Pushing 
this down into HB itself seems quite problematic to me, given the huge 
variety of types of compatibility decompositions, some of which require 
some kind of additional styling to avoid corrupting the intended meaning 
of the data.

JK