[Wasabi] Kicking of the Metadata spec - brainstorm

[jamie]

On Wed, 2007-02-21 at 08:54 -0500, Joe Shaw wrote:
> Hi,
> 
> jamie wrote:
> > (3) is the safest and best way - you get a richer service that allows
> > all apps to share metadata and stay up to date.
> > 
> > (2) is great in theory but falls over in practice for loads of reasons
> > (mainly need for fallbacks). XMP sidecar is better and really just needs
> > GVFS and KIO slaves to support it transparently (IE to auto copy the
> > hidden sidecar file whenever you copy/move a file)
> > 
> > (1) fails  for same reasons (2) does - lack of compatibility means
> > having to have fallbacks in place for file types that dont have embedded
> > metadata. (if that fallback is going to be XMP sidecar then why bother
> > with anything else?)
> 
> Sorry, I must not have been clear in my previous email.  The entire 
> point is to have fallbacks, because it gives you the richest experience. 
>   You try (1) [inline metdata] and if it fails or doesn't make sense, 
> you try (2) [extended attributes].  Only then if it fails do you do (3).

for an app that wants to fetch metadata it seems to be a lot of hoops to
jump through.

What would be fine is perhaps (1) + (3) and (2) + (3) so that an app can
easily get the metadata regardless of where its stored (via (3))

though I prefer XMP sidecar + (3) with (3) backing up all user defined
metadata there that way an app has a single gateway to all metadata.

> 
> XMP sidecars are a possibility (probably as step 2.5), but they have the 
> same problem as a centralized database: they don't follow the file 
> around.  If you copy a file, rename a file, or send it as an email 
> attachment, if you don't know to do the same with the sidecar you'll 
> lose it.  gnome-vfs/gvfs/kio support for this helps the issue, but it's 
> not as tightly coupled as xattrs.

I find it the other way round : xattrs fails as soon as you try and copy
them off the volume. They simply dont work on the network, in
attachments, to floppy/CD or USB stick etc. XMP sidecar seems to be the
best choice here (if we have vfs support for them)

> 
> And you say that (1) and (2) fail in practice, but this just isn't true. 
>   F-Spot stores its tags as XMP data in image files quite successfully. 
>   Rhythmbox, sound-juicer, and Banshee have been putting id3 tags in 
> MP3s forever.  As for xattrs, we've been using them very successfully in 
> Beagle for over 3 years (with a transparent fallback to an sqlite 
> database when it's not possible).  So all of this is possible to do and 
> give the user a good experience.

yes with certain file types - my point is its not a *universal*
solution.

> 
>  > Also there is notifications which you need a daemon for - its why
>  > gconf is a daemon!
> 
> Notifications are tricky, but I think it might be possible to do them 
> within a library using inotify (or FAM or other appropriate method). 
> gconf is a daemon because (a) mechanisms like these didn't exist and (b) 
> because absolutely everything touches it, and it doesn't make sense to 
> continually reparse its XML files.

not really a notification should pass the new value for the specific
metadata in the signal so apps dont need to work out which metadata has
changed or force them to refresh everything. I dont think thats possible
without a daemon? (you would need a copy of the old metadata to work out
whats changed)

(KConfig has failed to implement notifications for exactly that reason)

> 
> With approaches (1) and (2), notification is trivial.  You get 
> WRITE_CLOSE event and ATTR_CHANGED events on the files themselves.  For 
> (3), you can watch the database file for changes.  (This means you have 
> to timestamp rows, but once that's done you can trivially SELECT for 
> changes newer than a certain time.  This is a good thing to do in any case.)

(3) would potentially be an enormous db - it wont be practical to do
that and as above places too much burden on the apps (again you need the
old metadata to work out whats changed). Its kind of going to extreme
lengths (and pain) to avoid having a daemon to me but then thats just my
opinion!

the other reason for a daemon is its safer on broken nfs if only one
process ever writes to the db at a time (that of course wont work if a
user has multiple sessions with a single shared nfs home directory
between them but people who do that on broken nfs are asking for
trouble!).

jamie