[RFC] Metadata access and storage

[Federico Mena Quintero]

On Tue, 2011-09-06 at 20:02 -0400, Michael Pyne wrote:

> What I talked about regarding Zeitgeist was based on browsing its website 
> right before I sent the email. My understanding based on that was that 
> Zeitgeist was a more full-fledged semantic framework than mere events. I can 
> see from re-reading the description that it does indeed claim to be more of a 
> semantic logging framework.
> 
> With that said it certainly talks about many features which would be useful in 
> a more generic semantic layer but you'd have to talk with someone more 
> familiar with the library to see how far away it really is.

Zeitgeist is not a generic metadata store; it's just a time-based log of
user-initiated events.

I'll paste a short description that I sent to gnome-shell-list a while
ago:

Zeitgeist is ~/.recently-used on steroids.  It is a service that keeps a
time-ordered log of things that happen during your session.  In
Zeitgeist's terminology, each of those things is an Event.  An event has
various properties:

        timestamp - when did this happen?
        actor - what application caused this to happen?
        subject - what is being referred to (e.g. a URI or a file)?

(There are other properties, but in this mail we'll only care about
those three.  For example, there are other properties that let you say,
"this file came from an attachment to a mail in Evolution", or "this PDF
got downloaded from that webpage", so you can go back to the file's
origin from a graphical journal view.)

Various things log events into Zeitgeist.  We monitor your recently-used
list and feed Zeitgeist with events created from the data in that list
(ideally apps would log directly to Zeitgeist instead of the
recently-used list, but this is done for compatibility).  There are
extensions or plugins or patches for apps to log their actions
into Zeitgeist.  (All of this happens through D-Bus.)

Zeitgeist thus stores a log of things like

        - What files have I visited? (recently-used)
        - What web pages have I visited? (Firefox extension)
        - What Tomboy notes have I visited? (Tomboy add-in)
        - What IM conversations have I had? (Telepathy extension)
        - What music have I listened to? (Banshee plugin)
        - What applications have I launched? (GIO magic)
        - What pushes have I done? (bzr plugin)

That's the logging part.  But the interesting thing is how you can query
that log.

Zeitgeist provides a D-Bus API for querying the log in various
interesting ways:

- Give me the files that I visited within this time range.

- Give me the files that I've used most frequently with this app.

- Give me the files that I've opened close in time to this other file.

- Give me the apps that I use most frequently.

Etc.  This is done with a clever API based on providing an "event
template" with various wildcards.  This is not only for files, but for
any kind of subject you can put in an event, of course - thus it also
works for Tomboy notes or Banshee songs.

In terms of implementation, Zeitgeist has an "engine" daemon which
exports a D-Bus interface for adding or querying events.  All
communication with the engine is via D-Bus.  The engine stores its
database of events in an SQlite database.


So, you can see that this is not a generic metadata store.  It *has*
some interesting metadata, like "what web page was this file downloaded
from?", but that's ancillary to the central thing, which is to store
time-based events of things you do.

  Federico