Specifying thumbnailers as a service

[Brian J. Tarricone]

On Tue, 02 Sep 2008 00:09:53 +0200 Philip Van Hoof wrote:

> On Mon, 2008-09-01 at 09:53 -0700, Brian J. Tarricone wrote:
> > On Mon, 01 Sep 2008 10:49:31 +0200 Philip Van Hoof wrote:
> 
> > > I think canceling is overkill. Making a thumbnail doesn't take
> > > longer than a minute (and a minute is an extreme case). Users
> > > don't cancel that.
> > 
> > Strongly disagree.  I have a laptop, and say I'm on battery power.
> > If I'm scrolling through a big list of images in my file manager,
> > and it's requesting thumbnails, they all get queued. 
> 
> Then you must tell the developer of your filemanager that he should
> only request thumbnails for visible items, and for items that are
> likely going to become visible soon. Instead of immediately all items
> in the opened directory.

Yes, but that still doesn't solve the problem since thumbnails don't
take zero time to create.  I can scroll down a list of 500 photos in a
file manager slowly enough so thumbnail creation is triggered for all
of them, but close the window before thumbnails for even a quarter of
them are generated.

> He could just prioritize the visible ones, and in a timeout check, if
> the window is not closing, request thumbnails for nearby invisible
> items.

Right, but you're assuming the user won't scroll around all that much.
That's a very unsafe assumption.

> No need for cancels ...

Still disagree.

> Just requesting all and canceling the unfinished ones when the window
> is closing, sounds like messy programming to me.

Nowhere did I suggest the file manager implementer would do this.

> I'm pretty sure most filemanagers are already doing such optimizations
> for their thumbnailing work. Maybe even more clever ones.

Yes, they're implementing all you've suggested above... including
things like cancellation of in-flight (and queued) thumbnailing.

> > Say I find the file I want near the end of the list (by file name,
> > not thumbnail) and close the file manager window... but the
> > thumbnail service keeps chugging along, uselessly creating
> > thumbnails for 10s or 100s of files, pegging the CPU and wasting my
> > battery life.  I wouldn't be particularly happy about that.
> 
> But then again, that wouldn't happen if the developer of the
> filemanger is clever and only requests thumbnails for items that are
> visible and perhaps after that starts with requesting the nearby
> items (and if the window is closing, stops sending more requests).

If the user is scrolling down the list slowly enough, potentially *all*
items in the directory might become visible and queued.  You can
*never* assume the user isn't going to trigger the worst-case
performance in your application.

> Also note that this specification doesn't require a filemanager to use
> it. A filemanager can still do its own "specialized" thumbnailing, and
> it can even put thumbnail cache in $HOME/.thumbnail (the availability
> of a daemon doesn't mean that nobody else can write in that cache).

What's the point of this if you're already starting a list of
applications that won't want to use it?  I'd say potentially the biggest
consumer of thumbnails on the desktop would be the file manager.  If
that's true (maybe it isn't, this is just my feeling), shouldn't a spec
covering thumbnails think about its largest user?

> > Cancellation could be easily implemented by returning an ID from the
> > Create request that can be used to cancel the operation.  On the
> > daemon side, canceling the request could be as easy as killing a
> > subprocess, depending on implementation...
> 
> Who says that the daemon will implement creating a thumbnail in a
> proces that can be killed?

I'm not.  But thumbnailers that don't offer great ways to cancel in
mid-progress would need to be if they want to support cancellation in a
reasonable manner (with a reasonably short delay).

> To be honest wouldn't that be a very good
> architec- ture for a thumbnailer service:
> 
>   Imagine the thumbnailer creating 15,000 processes in case the 
>   filemanager's developer didn't do it right and therefore did request
>   thumbnails for all of the 15,000 items in your "Photos"-folder?

Oh please.  You're picking the most naive, crappy implementation idea
as your rebuttal?  So I take it you'd expect that the analogous
in-process thumbnailer would create 15,000 simultaneous threads to
create its thumbnails?

Any competent implementer of the main daemon service that intends to
support out-of-process thumbnailers would queue requests and only have
a small number of them in flight at any time.  Just like any competent
implementer who only wants to support in-process thumbnailers would
queue them and not run them all at the same time (in threads or
otherwise).

And also note that, with this kind of queuing architecture,
cancellation for 'jobs' that are still in the queue and haven't started
yet is trivial to implement.  Even if you don't want to support
fine-grained cancellation or out-of-process thumbnailers, if you have
15,000 requests queued and the thumbnailer is currently processing
requests #22, #23, and #24, a cancellation at that point would avoid
unnecessarily creating thumbnails for 14,975 of those.

	-brian