[gst-devel] On the plugin cache

[Jan Schmidt]

On Sat, 2008-11-08 at 10:25 +0100, Simon Holm Thøgersen wrote:
> [ Resending as this one didn't seem to reach the list. ]
> 
> ons, 05 11 2008 kl. 21:51 -0500, skrev Behdad Esfahbod: 
> > So how should the cache work?  By comparing the timestamp of each plugin dir
> > to the recorded timestamp of that dir in the cache.  One must compare
> > timestamps for equality, not for being more recent as that is prune to clock
> > skew false negatives.
> 
> I completely agree with you Behdad that excessive work is being done
> when stating files and not just dirs. However, even with the current
> design it is not where most of the time is spent.
> 
> The following is a profile of my laptop (Intel Pentium-m @1.5GHz)
> running 'gst-launch-0.10 --gst-disable-registry-fork' with 157 plugins
> present:
> 
> total 22 ms
>   linking libs 4.1ms
>   gst_init 17.9 ms
>     misc 3.3 ms
>     loading registry.i686.bin 12.4 ms
>       crc 1.5 ms
>       creating elements 10.9 ms
>     stating 2.2 ms

That raises an interesting point - it wasn't clear from Behdad's email
if his system is using the (new) binary registry cache format, or the
(old and slower) xml registry.

I see something like those timings here on my machine, including the
2.2-ish ms to stat things, on a machine with 174 plugins, 802 features.
(2.33Ghz Core 2 duo)

> I have a very simple patch that almost halves the time spent stating
> that I did almost a year ago but never got around to posting; I'll do
> that now.

Yes please :)

> Back then I also looked at speeding up the creation of elements.
> According to a comment I made back then it should be possible to reduce
> the time with 75%. I'm pretty sure that some of the changes broke ABI,
> but I'm not sure whether those that did were responsible for the
> speedup. I might try to port the work I did back then to the current
> gstreamer.

What was your approach for that?

> Before I look more into this, it would be nice if others posted similar
> profiles for their systems that could confirm this distribution. To
> generate my profile I added a bit of custom debug for the crc part, but
> the following should be sufficient for the rest.
> 
> export GST_DEBUG=3
> time /usr/bin/gst-launch-0.10 --gst-disable-registry-fork
> 
> The value of doing the crc check seems pretty dubious to me btw; if
> you've got disk corruptions there are plenty of other ways your system
> could malfunction. It should be pretty easy to make it optional at load
> time though.
> 

I'm not sure what the rationale was for adding a CRC to the binary
registry format originally. It might have been done as a sanity check to
detect partially-written registry files, so they can be rebuilt without
crashing every GStreamer app.

J.
-- 
Jan Schmidt <thaytan at noraisin.net>