[Fontconfig] fontconfig cache problems with `FcConfigAppFontAddDir`

[Werner LEMBERG]

Folks,


this e-mail is cross-posted to two lists since the problems are
relevant to both LilyPond and FontConfig, and I hope that I can get
comments from both sides.

I've just had a very nasty experience with FontConfig on my GNU/Linux
box that cost me hours of checking my own changes for bugs before
identifying the real problem...

[My box has FontConfig 2.12.6, and maybe this is fixed in newer
versions already.]

Consider the following situation.

1. Compile and install the current git version of LilyPond.

2. Compile a simple document like `\markup \number 1♭` that triggers
   LilyPond's Pango interface for handling Lilypond's 'Emmentaler'
   font (character '1'), but also contains a character not part of
   'Emmentaler' ('♭') so that FontConfig's font fallback mechanism
   takes action.

3. Add character '♭' to the 'Emmentaler' font, compile, and install
   LilyPond again (into the same directory).

4. If I now recompile `\markup \number 1♭`, I don't see the new glyph!
   Instead, I get exactly the same output as before.

   To fix this I had to delete the two newest FontConfig cache files
   (on my box they are in `~/.fontconfig/`), thus triggering a
   re-caching action.

I'm not sure whether this is a FontConfig or LilyPond bug (or both):
We temporarily add LilyPond's font directory to FontConfig to make
qPango find 'Emmentaler' fonts.  FontConfig then caches the data of
this temporarily added directory in its cache files.  (Among other
things, this cache holds information on which character codes are
covered by a font.)

In the next LilyPond run, with updated LilyPond fonts, adding the
directory temporarily to FontConfig apparently doesn't trigger a
re-cache: FontConfig sees that there is already information on the
'Emmentaler' fonts in the cache file and simply uses it.  This looks
like a bug, since FontConfig seems not to be able to recognize that
the newly installed fonts are different from the old one.

I can imagine two work-arounds.

1. Make LilyPond disable FontConfig caching of its own fonts.  I'm not
   sure, however, whether FontConfig provides an interface for that:
   Looking into the documentation I see that `FcConfigAppFontAddDir`
   (which we use) can't control cache usage.  However, there exists
   `FcConfigAppFontClear` to uninstall LilyPond fonts after use – I
   guess this function also removes the corresponding fonts from
   FontConfig's cache.

   But even if this works it probably has a severe performance impact
   because all fonts in LilyPond's font directory would have to be
   analyzed anew if LilyPond is run again.

2. At installation time, make FontConfig scan LilyPond's font
   directory and add a cache file to this directory.  AFAICS,
   FontConfig is able to read cache files from multiple directories.
   At execution time of LilyPond, with some new code, its font
   directory could be then added as another cache directory.

I favour option 2 since it would avoid the abovementioned problem with
FontConfig (i.e., not rescanning a directory added with
`FcConfigAppFontAddDir`).

Of course, there is also option

3. Do nothing.  Expect developers to manually clear the font cache if
   they are meddling with LilyPond fonts.

Comments?  Recommendations?


    Werner