DVFS and Metadata

[Jamie McCracken]

Timo Stuelten wrote:
> Hi!
> 
> Does it make sense to integrate some Metadata-Query-Thing (MQT) like 
> Spotlight/OS-X into the VFS-Layer? If reading or creating new files, that 
> MQT could then read contents, structures, metadata, whatever on-the-fly 
> without the need to create copies. 
> Or is it a completly different thing and listening to VFS-events and 
> rereading stuff suffices? 
> Or is it -- even while in the same pipeline -- completly seperated from 
> the VFS File Access API and should be plugged into the VFS via some 
> "I want all streams from the VFS"-API?

It should be plugged in as follows:

DVFS can check if stuff is up to date by comparing a folder's last 
changed timestamp with the one stored in the metadata DB - If they are 
identical then DVFS can provide metadata direct from the DB whch means 
something like nautilus could become super fast as it wont have to touch 
every file whenever you load a folder (IE it wont have to stat 
individual files in that folder cause it can get a folders contents, 
thumbnails and properties via the stored metadata).

The metadata stuff would be a seperate daemon so DVFS would need to tell 
it to update stuff if necessary (hopefully with Inotify working smoothly 
that wont happen too often).


> 
> Background: I want some mechansim which does not only create some fulltext 
> index for my text files and knows a few metadata types. 
> Instead, if I store my new maya-model, that MQT mechanism should extract all 3D-objects from the maya model (via 
> functional 3D-descriptions of things like chair, table, car, ...) for 
> which I want to build some topic map like index (Chairs are in the files 
> a, b, c, ...). Or think of files with MPEG-7 or RDF/OWL descriptors. The 
> MQT must store data of previously unknown semantics then.

To index nd extract metadata you would need filters for certain mime 
types so that is no problem (it would be expandable with flter backends).

>  
> That has nothing to do with file-access but maybe file-retrieval. It will 
> not work with some key-value metadata records. It will not work the 
> way spotlight works (which, afaik, builds some indexes for full text 
> and some predefinied metadata keys.)
> 
> It may need some techniques from the semantic web. So 
> it is total overkill for a VFS < version 42.0 probably. But 
> nevertheless: should it be possible to integrate or plug some non-trivial 
> MQT into the VFS? Must it be reachable via the VFS-API? Or does no app want 
> this, but only the desktop search tool and so the integration is some 
> implementation-depend question or is totally independend from VFS 
> and it should be called [A-Z]*(MQT)? instead and queries the DVFS, 
> DConf, D.* asynchronously in late 2017?
> 
> I read something about GStorage and some projects around, which may try 
> something similar. Is there some expert around?

Gnome storage is a completely different kettle of fish - it decomposes 
files into a DB and then recomposes them when you access them. Its very 
difficult to get such a system to work without eating your files though 
so I wouldn't trust it :)

> 
> It's still the requirements gathering stage, isn't it? :)

Yes.

jamie.

> 
> -Timo-
>