[Telepathy] thinking about a new log format for telepathy-logger

[Danielle Madeley]

On Sun, 2010-03-14 at 20:22 -0500, Matt Rogers wrote:
> On Sunday 14 March 2010 08:10:11 pm Danielle Madeley wrote:
> > So Telepathy Logger currently logs things in something not unlike
> > Empathy's XML log format. Although XML has some advantages (like being
> > able to generate logs using XSL), it seems fairly sub-optimal for the
> > efficient storage of logs.
> > 
> 
> How do you come to this conclusion? I understand that XML isn't really all 
> that compact in terms of bytes used. Is that where the problem lies?

The biggest problem is the parsing speed, which you would not call
stellar. It would also be nice to have an mmap'able format.

> > Solution #1: serialising binary structs

> urgh. Please no. It's so 1980's. I would hate to be in the shoes of having to 
> write a conversion routine for when the struct that gets written is changed.

Yeah. I agree, this alone sinks it.

> > Solution #2: binary tag-based thing (similar to Apple DMAP)

> Does custom matter if it's the best solution for the job? This is a pretty 
> nice format.

That's really the question isn't it. It only matters in so much as there
are no other tools that exist for the format (not that there seem to be
any good tools that exist for EXI, etc. either).

> > Solution #4: sqlite

> Is XML really a problem if we could put something like Tracker or Nepomuk with 
> it and then let those various semantic desktop services index and search logs? 
> Granted you still have the inefficent storage problem (in terms of bytes 
> used), but I'm not sure if that's really an issue.

I think people have tried logging using Tracker RDF and again it really
didn't scale. I doubt Soprano would scale any better.

There is a solution #5, which is to use some other database format. I
wonder if qdbm (http://qdbm.sourceforge.net/) could be used.

-- 
Danielle Madeley
Software Developer, Collabora Ltd.                  Melbourne, Australia

www.collabora.co.uk