[Xesam] Metadata Storage Daemon

Sebastian Trüg strueg at mandriva.com
Tue Jan 15 14:26:47 PST 2008


On Tuesday 15 January 2008 21:47:35 Jamie McCracken wrote:
> On Tue, 2008-01-15 at 21:02 +0100, Mikkel Kamstrup Erlandsen wrote:
> > On 15/01/2008, Jamie McCracken <jamie.mccrack at googlemail.com> wrote:
> > > On Tue, 2008-01-15 at 16:36 +0100, Mikkel Kamstrup Erlandsen wrote:
> > > > On 15/01/2008, Sebastian Trüg <strueg at mandriva.com> wrote:
> > > > > On Friday 11 January 2008 23:54:07 Mikkel Kamstrup Erlandsen wrote:
> > > > > > I have one question that I have not really thought through myself
> > > > > > yet...
> > > > > >
> > > > > > Should the storage monitor files? Fx Should the store drop data
> > > > > > on files that are deleted? This makes good sense if we talk about
> > > > > > tagging, since I don't want to list files that had tag "foo" back
> > > > > > when they existed (apps could do a stat-dance to check existence,
> > > > > > but i would rather avoid that).
> > > > > >
> > > > > > Or more generally updating internal structures if parts of graphs
> > > > > > are modified in a way that must be propagated for the layout to
> > > > > > remain consistent (fx clean up dead links[1]).
> > > > >
> > > > > Actually in nepomuk I do this independently and I think this is
> > > > > solution to go for "these days". The storage service only provides
> > > > > the actual data while another file monitoring service handles
> > > > > updating it.
> > > > > This allows for an easy combination of different implementations
> > > > > and simple replacement with more advanced components. It also
> > > > > allows different services for different storage locations such as
> > > > > NFS or USB sticks or whatever. I think nowadays questions like
> > > > > these should very often be answered with "make it modular".
> > > >
> > > > Yes, I tend to agree on this. The indexer is already watching files
> > > > and maybe other sources. If other programs store data in the storage
> > > > it is probably easier for them to just update it manually instead of
> > > > having to provide a monitor-plugin (which again we would have to
> > > > standardize in a cross-platform way in Xesam, eeeks).
> > >
> > > if you want it to "just work" then it needs to be integrated with
> > > indexer otherwise it would require integration with all clients
> > > including vfs, email apps and anything else which might prove difficult
> > > and or overlap with each other.
> >
> > I am not sure what we want to make "just work" here ... :-)
> >
> > If it is automatic updates of index + index reflects changes from
> > storage that where made while the indexer was not running; I think we
> > could achieve this with a proper dbus signalling system and good time
> > stamping.
> >
> > Or is there something I am missing here?
>
> well what happens when you move a file?
>
> In tracker all the metadata moves with it (provided tracker is running
> when you move it of course)
>
> with a dumb metadata daemon you lose it (all the metadata still points
> to the old file) unless something keeps it in check (which as i said
> means changing vfs, email apps etc to do this)

That is handled by another service. I don't think it is up to the indexer or 
the store to watch files (well, maybe the indexer to update data, ok, sure). 
And in the end even in your combined solution you have two components 
internally. So for the sake of decoupling and cleaner APIs we should define 
it as different services from the beginning.

> even if you disable indexing in tracker it still does file monitoring so
> that user defined metadata moves or is deleted as files are
> moved/deleted hence it "just works". A dumb metadata storage daemon
> would soon get out of sync

I think you missed the idea of the extra service before. And the indexer does 
not create all the metadata as Mikkel already pointed out. The simplest thing 
is a tag or a file rating. This has nothing to do with the indexer. Thus, 
from a design point of view it does not make sense at all to have the indexer 
update this data.

Cheers,
Sebastian


More information about the Xesam mailing list