A common VFS and a Common conf-system [Part II]

[Waldo Bastian]

On Tuesday 01 March 2005 16:48, Sean Middleditch wrote:
> As promised, here's some more notes/ideas.  Still dead tired due to only
> four hours of sleep, but maybe I won't babble too uselessly.

First of all thanks for your efforts in this area. It's refreshing to see some 
ideas that actually make sense when it comes to vfs.

> My original idea was to *NOT* have the backends run in the library or
> host app itself, but instead to have a per-user daemon that handled all
> of the actual VFS work, and then the library would simply be an API that
> made it easy for an app to talk to the daemon.

Yes, I think this is a good approach. We may want to put some thought in 
whether we would want to have a single-threaded daemon process, a 
multi-threaded daemon or a single process deamon with additional worker 
processes.

> That said, the daemon itself should be an implementation detail of the
> library.  Users of the library should not be forced to even consider
> whether there's a daemon doing the real work or not; all they should
> think about is their app.  Additionally, there are backends where the
> daemon might not make a lot of sense, or cases where admins or
> developers might prefer in-process backends over the daemon. 

ClearCase and I believe also SELinux may present the application process with 
a slightly different environment in comparison with the daemon process, in 
such cases in-process (or at least forking off from the application process) 
may be needed.

> The differentiation between not-supported and default has an interesting
> possibility.  Many VFS operations are in truth specialized combination
> of other operations.  Move could be implemented as a copy followed by a
> delete.  Copy could be implemented as a read from the source to a write
> to the destination.  In many cases, a backend can be provide a more
> efficient version - network file systems often have a copy or move
> operation that doesn't need any round trips to the client and can
> possibly use fast-paths on the server like the rename syscall.

There will also be cases of "partially supported". E.g. a ftp backend will be 
able to provide "copy" functionality if the host and user credentials are the 
same for source and destination, other backends may even be able to copy 
between different hosts (and skip the current machine altogether).

In order to be more efficient (think highspeed networks) it may also be worth 
to support "copy to file" and "copy from file" in the backend so that you 
don't need to pass the data around to another process or in buffers, but can 
use something like sendfile().

In KDE we have ".protocol" files that list capabilities that describes certain 
operations that a protocol supports. We also have the vtable like approach 
where methods can return "not supported" and then we do something else. The 
default implementation returns "not supported" everywhere. In KDE the main 
reason for the capabilities is that we wanted to extend existing behavior in 
a compatible way. E.g. originally we only called the "copy" method if host 
and user credentials match between source and destination. We can't then just 
start calling the "copy" method for copies between different hosts because 
the existing backends will not be expecting that and may fail to handle such 
situation correctly. With a capability the backend can indicate that it can 
support such copies and we will be able to feed a backend only those kind of 
copy operations that it will actually be able to support.

Bottom line: 
* We should plan for different gradations of "support".
* vtables alone are most likely too rigid for that.
* Some form of capabilities (a method that returns the capabilities?) would be 
useful.

I wonder if there is truly a need for both "not supported" and "use default 
method". Is there a case where you would want to return "not supported" if 
there was a default method available? In KDE we only have "not supported" and 
we interpret that as "use default method" the end result automatically 
becomes "not supported" if we run out of default methods.

> However, some backends don't have the ability to provide those services,
> or developers of backends may not have the time or inclination to write
> code for those operations.  In these cases, I think it would be best
> that the library provide default versions of the operations that
> aggregate simpler operations.  The default move function would do copy
> and delete.  The default copy operation would start up a read/write pair
> and essentially stream the file to the second copy.

Yes, that what we do in KDE (in kdelibs/kio/kio/jobs.cpp to be precise). 
Having read/write/delete is enough to do move :-) Or get/put/delete as its 
called in KDE (showing the original focus on http)

> A final cool feature that the library can handle would be interactivity.
> The biggest use-case here is authentication information.  Again, I
> believe it best that a daemon be used such that the individual apps
> don't have to handle the GUI, or have access to the information entered
> into the GUI (like passwords).
>
> The D-VFS daemon could get a request to access, say,
> http://idisk.mac.com.  That requires authentication.  It uses D-BUS to
> ask the authentication helper to ask for a username and password for the
> URI.  User enters it, it gets sent to the daemon, access continues.  Due
> to the asynchronous nature of the client-side API, the application sees
> no difference between waiting for authentication and a really slow
> network connection.

Yes, this is the approach that we do in KDE basically. Our backends ask a 
central deamon for a password and the daemon then either fetches it from the 
keyring, from local cache or it pops up a dialog. Since we use multiple 
processes for our backends in KDE, we need a local cache to store 
authentication information so that we don't need to popup multiple dialogs if 
two different backend processes access the same site as part of the same 
operation. (Typical example: fetching a html page and its css style sheet)

And you also need a way for the backend to say to the authentication daemon 
"that last password that you gave me doesn't work"
(This is a problem with FTP sites btw, where it's hard to decide whether the 
password is wrong or the FTP site has reached some sort of limit)

> We can take this a step further, though, if desired.  Look back to the
> idea above about restricting what apps can do and access to sensitive
> data.  Sure, with a daemon, we can be sure that Malicious App Foo can't
> get my password for idisk.mac.com, but it can still grab, delete, or
> edit files there.  With interactivity, we can specify that apps outside
> of /usr must get user permission to access shares; maybe even that apps
> in, say, /tmp are just flat out denied.

Is it possible for the daemon to reliably establish the executable path of the 
connecting process?

Cheers,
Waldo
-- 
bastian at kde.org   |   Free Novell Linux Desktop 9 Evaluation Download
bastian at suse.com  |   http://www.novell.com/products/desktop/eval.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.freedesktop.org/archives/xdg/attachments/20050301/83e3c4bd/attachment.pgp