<div dir="ltr"><div><div><div><div><div><div>Thanks Raimund for looking into this! </div>There are four usecases we need to consider: </div> - Normal Linux / other Unix-like. We can assume high-precision timestamps are available and take care of stuff, </div> - Linux / Unix-like system with network file system. Any issues there with your proposed patch? </div> - Linux / Unix-like system with FAT or other low-precision timestamp file system. Do we care much about that? I believe we've fixed bugs for those systems before. I just am not sure how much we care about small races there. </div> - Windows. I'm fine with leaving some known races there alone. </div>behdad </div><div class="gmail_extra"> <div class="gmail_quote">On Sun, Mar 13, 2016 at 1:12 PM, Raimund Steger <<a href="mailto:rs@mytum.de" target="_blank">rs@mytum.de</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Fri, 11 Mar 2016 19:16:57 +0900 Akira TAGOH <<a href="mailto:akira@tagoh.org">akira@tagoh.org</a>> wrote: > f44bfad2 could be reverted perhaps but need to test carefully before > doing that. unfortunately I don't have a time to do that so far. I think that might be a good idea. I've now read the linked Fedora bug again [1] and as far as I understand (with only little insight in Fedora's history with this problem): The race condition surfaced when installing urw-fonts as part of an enblend (or similar tools, in terms of dependencies) build, in that the .../Type1 directory cache somehow had an inadequate timestamp after postinstall. This was attributed to parallel invocations of fontconfig's caching routines. Now I've looked at the dependencies of enblend. Indeed I see some stuff that might pull other fonts in, for example gnuplot which depends on DejaVu fonts. Nowadays this probably doesn't take too long so all of it could really take place within seconds. I've also downloaded the attached test case and linked it with fontconfig-2.9.0, 2.10.0, and 2.11.9x (current Git). Indeed, with all earlier versions (than 2.11.9x) I can see the problem; the file 01.pcf, which is added while the other stuff is scanned, isn't picked up. However, if I check the mtimes in /tmp I see that everything happens in the same second: sun2:fontconfig)ls -E /tmp/fcbz69845-fonts-XXgyaO2j total 216 drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.286523696 +0100 0/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.331396817 +0100 01/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.286844908 +0100 1/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.288598316 +0100 10/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.288793584 +0100 11/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.288986704 +0100 12/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.289179054 +0100 13/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.289372500 +0100 14/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.289566288 +0100 15/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.289756148 +0100 16/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.289947012 +0100 17/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.290156261 +0100 18/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.290349106 +0100 19/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.287043469 +0100 2/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.290539282 +0100 20/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.290731146 +0100 21/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.290920807 +0100 22/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.291111970 +0100 23/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.291304851 +0100 24/ drwxr-xr-x 2 rs rs01 180 2016-03-13 20:42:25.291496406 +0100 25/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.287238447 +0100 3/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.287432511 +0100 4/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.287626802 +0100 5/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.287820816 +0100 6/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.288018928 +0100 7/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.288210736 +0100 8/ drwxr-xr-x 2 rs rs01 179 2016-03-13 20:42:25.288402321 +0100 9/ I then tried to revert f44bfad2 and also the FcDirCacheLock/Unlock part of ad9f5880502c9a3f8e7f8919336888ee056f17ba (attached) and tried the test again. The latter is because I have the feeling it's not strictly required either. The cache files have their individual locking and even if the dir isn't locked, as Keith argued (and I can find no flaw there) any later fc-cache /will/ pick the stuff up. Now I see that whenever I also remove the nanosecond stuff, the test fails. When I leave it in, it works. Of course that's no extensive testing or anything. But it might just be that the finer mtime is now good enough. Raimund [1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=921706" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=921706</a> <div><div class="h5"> > > On Fri, Mar 11, 2016 at 8:26 AM, Behdad Esfahbod <<a href="mailto:behdad@behdad.org">behdad@behdad.org</a>> wrote: > > Any resolution here? Current master's behavior of stating all fonts is > > definitely very bad. Anyone volunteer to clean up the cache code a bit? > > > > On Mon, Jan 18, 2016 at 7:09 PM, Akira TAGOH <<a href="mailto:akira@tagoh.org">akira@tagoh.org</a>> wrote: > >> > >> On Mon, Jan 18, 2016 at 5:54 PM, Keith Packard <<a href="mailto:keithp@keithp.com">keithp@keithp.com</a>> wrote: > >> > Akira TAGOH <<a href="mailto:akira@tagoh.org">akira@tagoh.org</a>> writes: > >> > > >> >> Not exactly. fontconfig doesn't stat everey directory at runtime but > >> >> relies on the subdir list in the cache to traverse. so if it isn't > >> >> outdated and missing some, it won't be picked up by fontconfig. > >> > > >> > How can it "miss some" if it checks the timestamp of the parent before > >> > relying on the list? > >> > >> Previously fontconfig only locks the file when writing out. the time > >> to write it out since bringing up fc-cache depends on the time of > >> collecting data and it takes more time if there are more targeted > >> files and directories. and likely to see a process finished first > >> which brought up later. > >> so I locked out between reading and writing to avoid it. > >> > >> >> Not sure but perhaps it is. my memory is already vague and not sure if > >> >> it had a nano second in the mtime. so better testing anyway. > >> > > >> > Hrm. Is fontconfig even using the higher resolution time when the file > >> > system supports it? > >> > >> Yes. well, very recently. not yet included in any releases. > >> > >> -- > >> Akira TAGOH > >> _______________________________________________ > >> Fontconfig mailing list > >> <a href="mailto:Fontconfig@lists.freedesktop.org">Fontconfig@lists.freedesktop.org</a> > >> <a href="http://lists.freedesktop.org/mailman/listinfo/fontconfig" rel="noreferrer" target="_blank">http://lists.freedesktop.org/mailman/listinfo/fontconfig</a> > > > > > > > > > > -- > > behdad > > <a href="http://behdad.org/" rel="noreferrer" target="_blank">http://behdad.org/</a> > > > > -- > Akira TAGOH > _______________________________________________ > Fontconfig mailing list > <a href="mailto:Fontconfig@lists.freedesktop.org">Fontconfig@lists.freedesktop.org</a> </div></div>> <a href="https://lists.freedesktop.org/mailman/listinfo/fontconfig" rel="noreferrer" target="_blank">https://lists.freedesktop.org/mailman/listinfo/fontconfig</a> <div class="HOEnZb"><div class="h5"> -- Worringer Str 31 Duesseldorf 40211 DE home: <a href="mailto:rs@mytum.de">rs@mytum.de</a> <a href="tel:%2B49-179-2981632" value="+491792981632">+49-179-2981632</a> icq 16845346 work: <a href="mailto:rs@interface-ag.de">rs@interface-ag.de</a> </div></div></blockquote></div> -- <div class="gmail_signature">behdad <a href="http://behdad.org/" target="_blank">http://behdad.org/</a></div> </div>