[Clipart] Re: [Cctools-cchost] Re: timing tests

[Victor Stone]

Victor wrote:
> I'm seriously 
> considering just writing specialized, tuned code for the record dump. 
> Basically a big loop with print statements outputting td's and div's.

Yea, that did the trick.

here's the numbers of the code currently on ccMixter for rendering the 
remix listing (detailed explanation of what the phases are below):

   includes: 0.2366 ( in sql: 0 0% )
   app init:  0.0277 (in sql: 0.0089 32%)
   perform action:  1.4631 (in sql: 1.006 69%)
   show:  1.0626 (in sql: 0.274 26%)            <========
   app done:  2.7916 (in sql: 1.282 46%)

Now here's this morning's numbers:

   includes: 0.2380 ( in sql: 0 0% )
   app init: 0.0223 ( in sql: 0.0061 27% )
   perform action: 0.5154 ( in sql: 0.3664 71% )
   show: 0.1796 ( in sql: 0.0033 2% )           <========
   app done: 0.9558 ( in sql: 0.3758 39% )

Rendering the page went from 1.0sec to 0.1sec, combined with the sql 
optimizations from this weekend the overall page delivery is 3x faster 
-- wahoo!

HOW I DID IT:

ccHost goes through several phases to build a page.

    includes:        includes every .php in sight
                     during this phase different parts of code
                     register handlers for every possible
                     incoming URL

    app init:        initialize global variables, log users in

    perform action: the registered handler for the current URL is
                    called and everything needed to render the page
                    is stuffed into huge arrays. Included in that
                    data are the names of template macros expected
                    to be used during rendering.

    show:           those arrays are all handed off, en masse, to the
                    template engine with the name of top level skin.
                    The skins are organized to dynamically invoke the
                    template macros called for in the data

One of the main reasons I chose phpTAL is because they made it easy to 
call BACK into your php code while executing a template macro. That can 
look something like:

  <tal:block define="foo php: call_into_cchost('edpick',time());"/>
  <div>${foo}</div>

If you look at cctemplates/upload.xml there is a macro called 
"list_files" It is big, gnarly, impossible to read and apparently 
extremely expensive for phpTAL to execute.

I replaced the innards of the whole macro a single php: call to a new 
function I wrote that matches the functionality but with 400 lines of 
simple, straight php code.

  <tal:block define="html php: mixter_list_recs($records);"/>
  <div>${html}</div>

My current thinking is have simple-skin continue to use the big ugly 
template because it is very general and to have ccmixter-skin (which 
nobody uses because it's broken and there for historical reasons) use 
this more specialized, tuned listing mechanism.

Another big improvement I made this morning is to cache the 'New 
Remixes' and 'Edpicks' listing over the left side of every page. I now 
write/read these to/from a temp file in phptal_cache. The drop from 0.27 
to 0.003 in sql usage during rendering will benefit every page on the site.


VS