[Fontconfig-bugs] [Bug 64766] Make fontconfig scanning faster

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=64766

--- Comment #29 from L A Walsh <freedesktop at tlinx.org> ---
(In reply to comment #3)
> I have no idea how this can be improved though.
----
Don't do it.

Only scan the files that are newer than the directory change time.
If no files are newer, no update.

If 1 file is newer (out of 3000), You copy the original file to a new 
file using a read/write size of 16MB-256MB.

Then add the new file at the point it needs to be added (and delete
it's old cached copy -- all in 1 write to the new file, then copy the rest
of the file in large chunks.

Reconstructing the font cache on my system takes 20-25 minutes if I 
have 32-bit and 64-bit libraries.  With just 64-bit libs, it's a bit
over 10, with 32 being closer to 15.  Cygwin (which has access to
all of window's fonts) takes 15-20 on 32-bit, haven't tried it since I
switched to 64-bit.

Note -- this wait usually happens at inconvenient moments, like when I
go to start 'gvim' to do any work.  Vs. having it be done in background w/out
me waiting on it.

No reason why I should have to wait for the fontcache to be rebuilt.
Look at the design of 'locate'.  It can take a half an hour to run on my system
(multiple TB disks).  The old database remains an is usable by users while
the new one is built.  Only when the new one is finished and ready for use
is it moved in to replace the old version.  

If you are scanning the dir and produce a copy so you are just changing
cached-content for new files, you'll be building your new copy in a
new location -- so any processes that start before it is built, will use
the old cache.

I'd like to point out, that font-data is architecture independent.  Turning
it into architecture *dependent* data that needs to be reconstructed for
32 and 64 bit on the same machine is a horrible step backwards in
compatibility.

Computing any type of digest will slow down things tremendously.  As an
example, I refer to my daily backups.  Using ***any type*** of compression
(lowest settings on gzip were last I tested, years ago, but lzop isn't
that much of an improvement), slowed backups down to between 5-10MB/s.
Without any compression -- ~200-250MB/s.  I gave up any compression and use  
5 rotating buffers of 1GB each. (notes in my backup script showed
1GB to be best (# 512m=> 220MB/s, 1g=>250mb/s, 2g => ~230MB/s).

Unless I'm sadly very mistaken, I'm pretty sure there are numerous ways
to speed up fontconfig.

Note -- some vendors are starting to install 1 font/directory.

That's only going to make the problem worse -- having to read through 3000
cache files in 3000 different dirs??   ARG!!!

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/fontconfig-bugs/attachments/20140519/5fd544f2/attachment.html>