A Standard for Thumbnailers

Tue Jan 9 15:12:34 PST 2007

On Tue, 09 Jan 2007 16:07:40 +0000 Erlend Davidson
<E.R.M.Davidson at sms.ed.ac.uk> babbled:

> 
> Carsten Haitzler (The Rasterman) wrote:
> > On Tue, 09 Jan 2007 12:08:46 +0100 Benedikt Meurer <benny at xfce.org> babbled:
> >
> >   
> >> Stanislav Brabec wrote:
> >>     
> >>> - Image files in public directories are thumbnailed many times:
> >>>   once for each accessing user.
> >>>       
> >> Sharing thumbnails might be a security problem.
> >>
> >>     
> >>> - Thumbnails on removable media are thumbnailed many times: once for
> >>>   each machine, where medium is accessed
> >>>       
> >> IIRC the standard already covers this, tho it's not implemented in Xfce
> >> and GNOME.
> >>
> >>     
> >>> - Thumbnail files are often larger than images itself, especially for
> >>>   jpeg images below 20kB in size.
> >>>       
> >> The software can detect this and load the JPEG directly.
> >>
> >>     
> >>> - There is a very small chance to detect deleted images and delete
> >>>   corresponding thumbnail.
> >>>       
> >> The file manager should clean up the thumbnails when deleting files.
> >>
> >>     
> >>> Use unique inode_number/volume_id instead of file_path.
> >>>       
> >> This will not work with certain FUSE file systems, that generate unique
> >> inodes on-demand, because it's likely that the inodes will be different
> >> each time the file system is mounted.
> >>
> >>     
> >>> Haw widely used desktop-neutral thumbnailing library understanding many
> >>> embedded thumbnails and providing thumbnailing for images without
> >>> embedded thumbnail.
> >>>       
> >> I think external thumbnailer as used by Thunar/Nautilus are better,
> >> because it's more flexible and avoids loading various libraries required
> >> to generate thumbnails for different formats into the processes.
> >>
> >> Using .desktop files to register thumbnailers instead of GConf should be
> >> fine for desktop-neutral usage.
> >>
> >>     
> >>> Allow jpeg thumbnails.
> >>>       
> >> Aside from maybe a few bytes saved per thumbnail, why would you want to
> >> do this?
> >>     
> >
> > it's nto a few bytes. for 128x128 sizes thumbs a thumbnail can go from a 20k
> > png to a 2k jpeg very very very easily. the jpeg thumbs invariably are
> > significantly smaller than the png ones (depending on the quality level you
> > are happy with). over the space of a few thousand thumbnails this make a BIG
> > difference.
> >   
> I have 6731 thumbnails in ~/thumbnails/normal.  This directory has a 
> size of 110MB, so that's about 15KB a file.  My home directory has 25GB 
> of used space, with all pictures, latex files, postscript files, pdf 
> files (which I have a lot of) being thumbnailed, so that seems quite 
> small to me.

depends how you do it. i use jpeg for thumbnails (within a container format)
and 128x128 thumbs end up 4-8k each. thats 3193 thumbs, 17Mb.

excluding any time used to load or scale src images i timed the overhead of
writing thumbs:
time to WRITE 3193 128x128 PNG thumbs: 115.55 sec
time to WRITE 3193 128x128 JPG thumbs: 7.59 sec

size of thumbs as PNG: 69M
size of thumbs as JPG: 17M

so... by doing nothing more complex than being more intelligent about my
thumbnail format choice - no complex code, no "amazing tricks", nothing beyond
what is already done with PNG, but instead doing it with JPG, I

1. spend 105 seconds or so less time in writing out thumbnails that i shouldn't
have spent (i.e. WASTE of CPU for no real gain).
2. I have WASTED 57M of disk storage pointlessly.

just by being adamant on using PNG as a format. if this were a discussion of
"but that extra cpu used and disk used buys you features X, Y and X" then we
have something to discuss. we are making a tradeoff. but we are not making a
tradeoff. we are simply wasting resources. these add up over 1000's of users.
thats extra disk IO time needed to load thumbs on retrieval, extra IO time to
save, extra processing time to save (and load incidentally). ALL we need to do
is SAVE as a JPEG file (and you can add alpha to JPEG if you so please so you
can't argue that only PNG can do this). You don't need to add any amazing code
- you already have wrappers for saving as JPEG right there - waiting to be
used. We can argue there is quality loss (which there is) but frankly - it's
imperceptible when you are scanning a list of thumbnails for your favorite
image (that's the point of JPEG).

efficiency MATTERS. The above timings were on a very high-end box (core 2 duo
laptop, 5600 with 2G of RAM). these numbers blow out significantly when you are
talking people in not so wealthy countries who are stuck on 500mhz p3's because
that is all they can afford (or less - e.g. the OLPC). why waste their valuable
disk space and cpu cycles? it's an incredibly simple thing to do (supporting
JPEG thumbnails) and has significantly MEASURABLE benefits (with the only
detriments being quality encoding loss - see above).

> Thumbnails are quite a high-end feature anyway - even on a fast computer 
> they take a little time to produce.

i disagree. if you do them right they can be generated VERY quickly even on a
low-end PC. I devoted a little bit of time and effort a few years back making
that trivial with EPEG - it's a wrapper around libjpeg intended for 1 thing
only. rapid jpeg thumbnailing. it makes use of the fact that JPEG's are DCT's
and you get "free" downscaling on decode by factors of 2, 4 and 8 times in each
dimension simply by decoding the DCT at a different output res (and throwing
out higher frequency DCT elements). it means for an incredibly faster decode
and thus thumbnail generation is sped up massively. you speed up the other half
of this work by using JPEG output for thumbnails. it makes a difference. a
perceptible and measurable one. people notice.

> >   
> >>> Thumbnails can use file system extended attributes.
> >>>       
> >> Not every file system supports that. What would be stored in the
> >> extended attributes anyway?
> >>
> >> Benedikt
> >> _______________________________________________
> >> xdg mailing list
> >> xdg at lists.freedesktop.org
> >> http://lists.freedesktop.org/mailman/listinfo/xdg
> >>
> >>     
> >
> >
> >   
> _______________________________________________
> xdg mailing list
> xdg at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xdg
> 

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster at rasterman.com
裸好多
Tokyo, Japan (東京 日本)