[Clipart] Recent fd.o disk IO is our issue

[Bryce Harrington]

On Fri, Jul 08, 2005 at 11:26:51PM -0700, Jon Phillips wrote:
> On Fri, 2005-07-08 at 09:34 -0400, Jonadab the Unsightly One wrote:
> > Jon Phillips <jon at rejon.org> writes:
> > 
> > > We should think about how to speed up the navigate engine and fix
> > > the security problem in the navigate and screenshot
> > > scripts. Thoughts?
> > 
> > The navigate script probably could be redesigned to read just one
> > index file, which could be generated as part of the release process.
> > That would also eliminate the path security issue as a side-effect.
> > Of course, the img tags would still link to the individual files, but
> > that's static web-serving stuff and should be okay.
> 
> Hmmm...what other options do we have as well. Seems like one big index
> file will still be accessed quite often. I wonder if also putting the
> system into sqlite or some other lite db system might speed up access
> and also decrease hard drive io. Also, what ways of caching (queries and
> the index data) could we use to speed up and decrease drive io in the
> navigate script?

This is exactly what databases were designed for.  ;-)

What we need to do is create a SQL schema for storing all of this
information.  The info can come from DMS or whereever else.  Greg has
written a Python script that'll do this, and I'm going to try to take a
good look at it today to see if I can make it fit with dms.

Bryce