Trash spec: directory size cache

[Diggory Hardy]

On Wednesday 17 April 2013 10:12:22 David Faure wrote:
> On Tuesday 16 April 2013 09:03:22 Alexander Larsson wrote:
> > On tis, 2013-04-16 at 00:15 +0200, Ryan Lortie wrote:
> > > hi David,
> > > 
> > > On 2013-04-15 18:47, David Faure wrote:
> > > > 16950 15803468 Documents
> > > > 2467 15803582 Another_Folder
> > > 
> > > One thing I forgot to ask for a clarification on earlier, and certainly
> > > something that we should spell out in the spec: what do we mean by
> > > 'size'?  Sum of byte-sizes of all files, or 'disk space used' sizes of
> > > all files and directories?
> > > 
> > > I guess the second one makes more sense, but the sizes you show here
> > > don't seem to be multiples of disk block sizes, which is usually the
> > > case for this type of sizes.
> > > 
> > > Your thoughts?
> > 
> > Sum of block sizes isn't a perfect measurement either. For one it
> > doesn't count the disk space of the directories themseleves, nor does it
> > handle things like tailpacking, hardlinks, etc.
> > 
> > However, it is more reliable than just the sum of the sizes (i.e. it
> > handles sparse files and many-small-files better), and its tractable to
> > compute, so I'd say go with it.
> 
> OK. Thanks everyone for the input.
> I have combined it all into the attached patch for the trash specification.
> 
> Note that I largely rewrote the non-normative algorithm compared to the
> initial email; please check.
> 
> If you find html-in-a-patch hard to read, you can look for "Directory size
> cache" at this page: http://www.davidfaure.fr/2013/trashspec_proposal.html

Hello,

Looks pretty good. Only thing that occurs to me is if too jobs are updating 
the directorysizes file simultaneously two problems are possible: a read 
clashing with a write or two clashing writes. Requiring atomic writes (i.e. to 
a temporary file followed by rename) avoids the first problem and is relatively 
simple. Clashed writes are probably not worth worrying about (worst case a 
slow/paused job overwrites a recent file with an old one, but this seems 
neither particularly bad nor likely from my POV).