DVFS and Metadata

[Jamie McCracken]

Timo Stuelten wrote:
> On Sun, 6 Mar 2005, Jamie McCracken wrote:
> 
>>It should be plugged in as follows:
>>
>>DVFS can check if stuff is up to date by comparing a folder's last changed
>>timestamp with the one stored in the metadata DB - If they are identical then
>>DVFS can provide metadata direct from the DB whch means something like
>>nautilus could become super fast as it wont have to touch every file whenever
>>you load a folder (IE it wont have to stat individual files in that folder
>>cause it can get a folders contents, thumbnails and properties via the stored
>>metadata).
>>
>>The metadata stuff would be a seperate daemon so DVFS would need to tell it to
>>update stuff if necessary (hopefully with Inotify working smoothly that wont
>>happen too often).
> 
> So you want to integrate metadata handling completly into the VFS? 

yes and it should support vfolders (IE I could have a vfolder that shows 
all word files regardless of where they exist on the filesystems)

> 
> 
>>To index nd extract metadata you would need filters for certain mime types so
>>that is no problem (it would be expandable with flter backends).
>>
>>...
>>
>>Gnome storage is a completely different kettle of fish - it decomposes files
>>into a DB and then recomposes them when you access them. Its very difficult to
>>get such a system to work without eating your files though so I wouldn't trust
>>it :)
> 
> Apple does exactly this in its storage subsystem, the same with hibernate and some other java based 
> frameworks. To get semantics from structures I think one has to decompose 
> files to understand them as documents. For the given 3D-Model example, 
> there is now way to handle models flat (i think). But that's not the point. 
> 
> What if the MQT changes from simple flat index to a
> structure-semantic-someotherfunkymechanism index? To get a thumbnail for a 
> folder, there is no need for an enhanced metadata-DB. To get all thumbs 
> and compose all with transparency there maybe is. But then you may want  
> something more powerful than "thumb.1=aaa.png", "thumb.2=bbb.png" as 
> key-value-pairs for the folder.
> And if you have this MQT, how to query it? Via getMetadata(folder:="abc", 
> key:="thumb.*") from the VFS-API?

I would imagine DVFS would have a getmetadata thing that returns a blob 
of key/value pairs or xml

The metadata will specify whats available (if it has thumbnails etc)

The actual thumbnail should come from the DB cause we want to avoid dot 
files at all costs for performance reasons

> 
> The question is: 
> Is some Metadata-Query-Thing bound to the VFS or does some MTQ-to-invent 
> uses the VFS?
> Should the VFS support various MQTs or use a simpler
> key-value-based approach with some file-type-metadata-extractor-plugins 
> which deliver common metadata using some predefined key and maybe some 
> spezialized too.
> Should it's features be part of the VFS-API? But then what about Mails, 
> IM messages., ...?
>

thats why we want a DB system where user data (be it emails, address 
books etc) as well as indexed data is available for searching.

Bear in mind there are three seperate things here:

1) DDS - desktop data server which provides a per user daemon that 
provides access to a database. Can be used as a backend for Dconf, 
storing metadata, address books, emails or anything else you want 
structured storage for.

2) Tracker - indexer/metadata crawler (also a per user daemon) which 
uses DDS for storage.

3) DVFS which would integrate with the tracker for all the 
metadata/indexing/vfolder stuff

  jamie.