[Xesam] Metadata Storage Daemon

[Mikkel Kamstrup Erlandsen]

On 15/01/2008, Jamie McCracken <jamie.mccrack at googlemail.com> wrote:
>
> On Tue, 2008-01-15 at 16:36 +0100, Mikkel Kamstrup Erlandsen wrote:
> > On 15/01/2008, Sebastian Trüg <strueg at mandriva.com> wrote:
> > > On Friday 11 January 2008 23:54:07 Mikkel Kamstrup Erlandsen wrote:
> > > > I have one question that I have not really thought through myself yet...
> > > >
> > > > Should the storage monitor files? Fx Should the store drop data on
> > > > files that are deleted? This makes good sense if we talk about
> > > > tagging, since I don't want to list files that had tag "foo" back when
> > > > they existed (apps could do a stat-dance to check existence, but i
> > > > would rather avoid that).
> > > >
> > > > Or more generally updating internal structures if parts of graphs are
> > > > modified in a way that must be propagated for the layout to remain
> > > > consistent (fx clean up dead links[1]).
> > >
> > > Actually in nepomuk I do this independently and I think this is solution to go
> > > for "these days". The storage service only provides the actual data while
> > > another file monitoring service handles updating it.
> > > This allows for an easy combination of different implementations and simple
> > > replacement with more advanced components. It also allows different services
> > > for different storage locations such as NFS or USB sticks or whatever. I
> > > think nowadays questions like these should very often be answered with "make
> > > it modular".
> > >
> >
> > Yes, I tend to agree on this. The indexer is already watching files
> > and maybe other sources. If other programs store data in the storage
> > it is probably easier for them to just update it manually instead of
> > having to provide a monitor-plugin (which again we would have to
> > standardize in a cross-platform way in Xesam, eeeks).
> >
>
> if you want it to "just work" then it needs to be integrated with
> indexer otherwise it would require integration with all clients
> including vfs, email apps and anything else which might prove difficult
> and or overlap with each other.

I am not sure what we want to make "just work" here ... :-)

If it is automatic updates of index + index reflects changes from
storage that where made while the indexer was not running; I think we
could achieve this with a proper dbus signalling system and good time
stamping.

Or is there something I am missing here?

> the other advantage of indexer integration is that the indexer already
> handles cases where data is modified or deleted even when its not
> running.
>
> my feeling is that duplication of effort and storage required with
> independent metadata daemon and indexer is too much and not practical
> (unless you want a dumb metadata daemon)

I think a "dumb daemon" is what is being proposed actually... Ie a
triple- or quadruple store with the ability to understand ontologies
and nothing more. Ie no monitoring, integrity checking,  or anything
fancy.

Also note that it might not be all file-metadata that should be stored
in the storage (that could possibly only be stored in the index). The
storage could possibly only hold non-implicit data (ie data somehow
generated by user actions) - subject to loose definitions of course.

Cheers,
Mikkel