[Fontconfig] [PATCH] Do not remove UUID file when a scanned directory is empty 1 x

[Alexander Larsson]

I just sent this mail, but was not a member, so pasting it here again
after joining:

On Tue, Oct 30, 2018 at 6:55 AM Behdad Esfahbod <behdad at behdad.org> wrote:
> On Mon, Oct 29, 2018 at 10:49 PM Keith Packard <keithp at keithp.com> wrote:
>>
>> Akira TAGOH <akira at tagoh.org> writes:
>>
>> > Not rare today. this has been made to improve flatpaks startup time
>> > because regenerating fontconfig caches always happened.
>>
>> I'd like to find a solution which doesn't involve writing files into the
>> font directories themselves, but I'm having trouble thinking of how that
>> might work without involving changes in the flatpaks themselves. All we
>> really need is some way to compute the right cache file name given the
>> path found inside the flatpak.
>>
>> Is the .fonts.conf file for the flatpak computed dynamically? Can we
>> adjust how that is built? What if that file contained an association
>> between the paths it uses and the paths used outside? We could then
>> construct the external path from the internal path and use that to
>> generate the right cache file name.
>>
>> Would we need more than one mapping per <dir> element in the font
>> configuration?
>>
>> Alternatively, a flat-pak using system could place a magic file in each
>> top-level font directory for fontconfig to use for this mapping. As that
>> would simply contain the name of the directory in the external
>> namespace, it would be easy to generate with a shell script even. At
>> least we wouldn't be creating and destroying these files each time
>> fc-cache runs, as we wouldn't need them in each new directory, only at
>> the top of each font tree.

So, let me explain the setup where flatpak runs into trouble first,
but I think this is a more generic problem which i'll come to later.

First of all, flatpak runs in a filesystem namespace, so we have a
different copy of fontconfig (the library and config), and different
pathnames for everything.
/usr/share/fonts inside the flatpak comes from the "runtime" (think of
it like a chroot) this way we can at least have some minimal fonts
guaranteed to be available. However, since exposing system fonts is
very useful and safe, we do expose the host fonts (the "real"
/usr/share/fonts) to the app as /run/host/fonts, and the host cache
directory in /run/host/fonts-cache (additionally as not all flatpak
have homedir access we expose /run/host/user-fonts and
/run/host/user-fonts-cache). Additionally, the fontconfig in the
runtime contains a fonts/conf.d snipped that adds these directories to
the font paths. Additionally affecting caches, all the flatpak files
(such as the fonts and the shipped cache in the runtime) are stored in
something called ostree, which normalizes mtimes (to 0) in order to
maximize sharing of files.

Now, this setup runs into some problems with the caches in the old cache system

First of all, since we index the cache by the pathname of the font
dir, when we exposing a font directory by a different name (such as
/run/host/fonts) we would not find the previously host-generated font
cache when running in a flatpak. This meant each flatpak app would get
a 20 second startup time the first time they are launched, plus its
own copy of a cache for the system fonts.

Secondly, since flatpaks always have fonts in /usr/share/fonts, but
they might have *different* fonts there (if they use different
runtimes, or it the runtime changes), the fonts kept breaking because
it thought that the font cache matched the directory (same path+mtime)
even though the font cache was generated for a different version of
the directory. Sometimes it even mixed up the host /usr/share/fonts
with the one in the runtime.

This kind of thing happens in flatpak, and all other container-like
setups (which are only going to be more and more common), but it can
also happen in other cases, like a cache in your homedir which refers
to a system font dir when you dual boot different distros, or a shared
NFS homedir used on many systems. Or a fedora atomic/silverblue
instance if you update to a new snapshot of the OS image. The earlier
case will work, but be inefficient (i.e. recreate caches each time)
due to the use of the mtime, but the silverblue case stores the entire
distro in ostree too, so the mtime will be identical, and your
fontconfig caches will not be detected as stale when you boot to a
different version.

So, to fix this we made each *instance* of a font dir unique, by
dropping a uuid file there. Then we can index the cache by the actual
physical directory. It also means the user is control of cache
invalidation, for example when creating a container image, chroot or
flatpak runtime you can decide on the uuid to use for your font dirs,
to ensure that old caches are reused or not, as needed. Obviously its
not great if we keep ping-ponging the uuid for empty directories
though, but I don't see how that is not fundamentally fixable, its not
inherent in the idea.

Working around this by trying to figure out the "real" host pathname
is not going to work great. Yes, we ship a custom config snippet with
flatpak to point it at /run/host/fonts, so we can amend that with some
extra info, but that will not really be enough. First of all these are
static files, but the "real" dir for e.g. /run/host/user-fonts is
dynamic. But even worse, this does not solve the second problem where
"different" /usr/share/fonts will all map to the same cache instance.

Fundamentally, the cache is caching a particular instance of a font
directory, not the current pathname for it, because what a pathname
points to changes between boots, between machines, and between
namespaces.