virtual filesystem ideas

[Ken Deeter]

Hmm.. looks like my last message may have been too big..
---

Here's something I threw together. I too am very interested an seeing
the writeups of the gnome-vfs and kio as I only know very high level
things about them.

Attached is some kind of architecture diagram and some text describing
various ideas in it. I guess its an elaboration on what I was saying
yesterday. I've also inserted the text part below. If anything maybe the
design goals part tries to sum up what we've talked about in this thread
so far.

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<


See attached diagram:

Legend:

  Dotted red arrows = "relaization" in the uml sense
  Green arrows = D-BUS transactions
  White boxes = software implementations
  Yellow boxes = D-Bus interfaces ('services')


Overview
--------

I think this design is essentially something along the lines of what
people have been thinking about, with the ability to 'hook' into
various parts of the process by passing references to services that
have well-defined interfaces. It's kind of like a call back, but it
uses D-BUS to let you define much more interesting interactions than
just a simple callback. Backends that use protocols that weren't meant
for async io can still be written in a fairly sync matter, it would
just have to block on certain things (like authentication).

Design goals
------------

* Simplicity: This is i think from the point of view of the
  application programmer and the backend developer. There should not
  be too many things that they have to worry about. From the user's
  perspective, it should just work (tm), but I think this specifically
  means that passing in URI's to various applications should result in
  regular and predictable consistent behaviour, including the kinds of
  authentication mechanisms presented.

* User-interaction decoupling: All the parts that require user
  interaction are hidden behind D-BUS services with specific message
  interfaces. This lets client apps not have to worry about anything
  unless they want to, and it allows desktops to provide integrated
  'default' solutions that can implement their own policies with
  regards to authentications, password caching, sessions management,
  etc. This allows for a situation like, if you run a KDE app in
  gnome, it should be able to use the proper gnome dialogs and
  policies.

* Client control: If a client wants to be really involved in the
  backend, like with browsers, it should be able to. My design so far
  allows clients to provide their own authentication services, as well
  as monitor backends in a way that may be specific to that backend.
  I suppose it would be useful to have a backend also specify any
  backend-specific control interfaces so that a client can have lots
  of control over a backend if it wanted to. Or maybe a mechanism by
  which a client can bypass the central vfs service altogether and use
  a backend directly and exculsively.

* Provide High level operations: I like the queryInterfaces() type of
  idea, save for that I'm not sure most application programmers want
  to deal with adapting to all kinds of different capabilities of
  various backends. So there should be someone in the middle that
  provides very high level operations and translates them to whatever
  capabilities a backend may provide, or tells you that they're not
  possible. This is probably hard to do very well, so it should only
  be done once.


VFS Service 
-----------

The Vfs service is at the core. It is sort of a management center that
provides high level vfs operations. This service would be running
regardless of whether a desktop was, so that it could provide vfs
services to console applications as well.

It also is responsible for activating back ends (or maybe this can be
done by d-bus directly), knowing what backends are available, providing
calls to query what kinds of backends are currently running, and to
provide a standard way to observe the state of the vfs in real time. I
think it would be nice to also bring something like fam into this
component so that we only need one interface to support monitoring
on all kinds of vfs's.

I think this would also be the place to put high level-> low level
operation composition kind of stuff. (i.e. implement recursive
directory delete as a series of change directories and deletes, as
many ftp programs do) I think kio does this, not sure about gnome-vfs.
But these highlevel operations are useful to everyone so they should
share an implementation.

A desktop when it runs should be able to tell the vfs to notify it of
events. I'm not sure exactly what kind of events, but there must be
something that the desktop would like to know in real time.


VFS Backend
-----------

This is where the actual work happens. This design tries to extract
all the GUI and user-interaction requiring stuff out of the module.

Each backend instance would be passed several service names as
parameters, one which implements an authentication service that is
specific to this backend (or i guess the interface could be shared if
i multiple backends had the same kind of authentication requirements)
and an observer interface, that would be capable of capturing events
within a backend that maybe specific to that back end.

For example i could say give me a sftp backend instance passing it
the gnome sftp authentication adapter, and the gnome sftp observer. The
authentication adapter would give me gnome dialogs that adhere
to gnome's policy of 'sessions' and password caching (maybe a dialog
wouldn't even pop up at all) and would also say, pop up gnome dialogs
to show the current status (like say if downloading af file) 

It seems like things like the status dialogs that come up when you do
a big copy in konqueror or nautilus should be able to be handled at
either the desktop level OR specifically by the client. You wouldn't
want little dialog windows popping up every time konqueror loads a
page, so an app like konq should be able to pass in its own observer
that could monitor the backend's status and update its own GUI in an
appropriate way. A console ftp app could also pass in its own observer
that just updates its own internal structures and displays information
in some way more appropriate for the console.

In the case that the app doesn't care what kind of authentication or
observer is used, there should of course be a call that just means
'use whatever is there'. The best way to do this is probably use the
'service ownership' idea in D-BUS so that when gnome starts up for
example, it would have something that takes ownership of a well known
auth adapter service name, org.vfs.auth.sftp or something similar. If
when the backend is created, not specific auth adapters or observers
are passed in, then it could just be hardwired to lookup the well
known service name.

The only thing I'm not sure about is, say you're just running in the
console, so no application may have claimed the default well-known
auth service. In that case, if you try to do something that requires a
service, the activation mehcanism will kick in, but you want it to do
something appropriate for the console. Is there a way in D-BUS that
you can say 'give me something that works on the console', or rather
is there a way to implement that kind of thing?


Issues
------

It seems kind of fundamental, but how does the client app actually get
the data it wants? In this design's organization, I suppose the back
end would deliver it to the vfs service, which would then deliver it
to the client application. I guess if D-BUS can do some kind of
shared-memory avoid-copying thing then this could be a reasonable
solution. Scheduling latency might be a problem with so many mutually
out-of-process interactions. In the case of network transfers, since
the network will likely be the bottleneck, maybe this isn't so much of
a problem. But if its dog slow, nobody's gonna use it.. especially for
'file://'





------

 +----------------------------------------------+
(  Ken Deeter (Kentarou SHINOHARA)             (
 )                                              )
(  "If only God were alive to see this.. "     (
 )                             -Homer Simpson   )
+----------------------------------------------+

-------------- next part --------------
A non-text attachment was scrubbed...
Name: diagram.jpg
Type: image/jpeg
Size: 17356 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/xdg/attachments/20030919/db17ce55/attachment.jpg