[Clipart] Searching and download graphics

[Greg Bulmash]

Then again, there's also the option of merely re-evaluating what has to 
be live and what can be almost live.

Since I add new batches of clipart to my library every 2-3 weeks, I 
wrote a script that generates HTML indexes on demand.  My master index 
page...

http://www.fundraw.com/clipart/

... and category index pages...

http://www.fundraw.com/clipart/categories/Princess_Clip_Art/00000181

... are all HTML flat files that I generate after a batch update to the 
clipart database.  Same with the XML indices used by the clipart menu 
within the FunDraw Art Studio application (the Flash-based design app 
that all this clipart can be used in).

The server can deliver the straight HTML/XML with a simple disk read (or 
memory read) instead of a larger database call.  That means lower memory 
usage, lower disk usage, and fewer CPU cycles consumed.

For an example on the OCL site, the tag cloud is eminently cacheable as 
a flat file.  Have a cron job update it every 6 hours as a flat file and 
deliver that.  If you're getting 3,000 hits a day on it, you're saving 
2,996 database operations daily (along with their associated increased 
memory and CPU demands).

With a good analysis of traffic patterns and user preferences, you could 
determine the best pages to set up as pre-generated on a periodic basis 
vs. being on-demand, and cut overall server load by double-digit 
percentages.

Get a good DBA to optimize the queries that remain on-demand and 
optimize the caching of the query results and you might knock that down 
by another double-digit percentage.

- Greg


Jon Phillips wrote:
> On Thu, 2007-05-10 at 11:02 -0600, Alan wrote:
>> I've been thinking.
>>
>> Your server is going to get hammered everytime people do searches on it
>> for particular graphics.   I like the idea of setting up a thumbnail
>> database and a search function to find particular photos, or you could
>> organize by type as well.
>>
>> Would it be possible to create a utility which downloads to a users
>> computer the thumbnail database and other search criteria?  This would
>> allow searches to take place locally.  When a user has decided what they
>> want, the utility will connect to the server and download what they
>> want.  The local utility could be set up to link to a complete download
>> or a multiple set of chosen graphics or a single picture at a time.
>>
>> This will reduce the workload on the server and allow users to browse
>> through the thumbnail photo collection at their leisure on their own
>> computer.
> 
> This is a great idea. Would you like to help realize this?
> 
> Bryce Harrington (on the list) has been working on Inkscape + Open Clip
> Art Library integration, and this is definitely something we have talked
> about.
> 
> Jon!
>