continued: Common-VFS proposal

[nf2]

Hi!

Sean Middleditch wrote:

>That being said, Norbert:
>
>First, what *is* with this "D-BUS won't work" deal?  That makes no
>sense, and you provide no explanation.  D-BUS is absolutely *perfect*
>for our needs.  Certainly something *more* heavyweight - like CORBA - is
>going to be worse.  VFS operations are very common, they need to be
>fast.
>  
>
Ok. I'll put questionmarks behind this statement. The reason why i wrote 
this is that
i thought that DBUS was not designed to transport gigabytes of data. But 
i really don't know which "wire protocol" for method calls (DBUS vs 
ORBIT) is leaner. The other reason was that ORBIT provides a nice 
framework - as far as i understand it provides a multi-threaded server 
for putting protocol handlers in the VFS daemon. This is used by 
Gnome-VFS AFAIK and might be nicer*) from design than running seperate 
processes like KIO or your proposal. I don't know whether a multi 
threaded server could be written with DBUS as well and if that wouldn't 
just be reinventing the ORBIT wheel.

*) It would be interesting to discuss the pros and cons of the thread 
(Gnome-VFS) vs the process (KIO) approach.

>The model I had in mind looked something like this:
>
>...
>
>The entire library and daemon suite would be designed for as low
>resource requirements as possible and keep as clean and UNIXy an API as
>possible.  The idea was to basically push this as a general, modern UNIX
>VFS API, with my ego letting me believe that in 5 or 10 years this might
>become a de facto standard across all UNIXes - which of course requires
>the same of D-BUS, which I also hope happens.  ;-)
>  
>
Yes- that would be cool - At least for desktop applications.

>The big problem is the event handling, especially when you realize that
>you're going to end up needing asynchronous I/O and probably some kind
>of monitoring as well.  Using D-BUS mostly solves this as it is already
>possible to integrate with any mainloop, has glib support to do so, and
>Qt integration is definitely possible.  A lot of CLI apps and such
>probably just wouldn't use any asynchronous operations.  The only
>problem comes from having local files handled in the library/application
>itself - local files would have distinct descriptors while all remote
>connections would be handled over one connection to the daemon;
>  
>
I don't know if the daemon is necessary for all network connections*). 
For sftp for instance that might not be necessary (if there is no 
connection limit). That's one of the things i like about Gnome-VFS. 
Certain modules go into the daemon and others stay in the client process 
(As threads in async mode AFAIK - please correct me!)

cat /etc/gnome-vfs-2.0/modules/default-modules.conf
cat /etc/gnome-vfs-2.0/modules/smb-module.conf (look at the [daemon] flag)

*) Although that might be nice, because it would be easier to disconnect 
them on demand.

>basically it's a problem of knowing which file descriptors to pass to
>select()/poll()/epoll/kqueue.  One solution is to provide an API to
>obtain a list of all descriptors in use by the VFS library.  Another is
>to simply state that each open file has on descriptor, and in the case
>of remote files use a separate daemon connection for each file (yuck).
>The final is to just document the problem clearly and punt to the apps.
>Since the problem only happens in asynchronous operations anyhow, which
>happen mostly in GUI mainloops, each of which would have its own
>simplified wrapping around the VFS, I'm leaning towards the final
>solution.
>  
>
The common main loop... I think commercial os'es have that... ;-)

>The bridging of gnome-vfs and kioslaves just is *not* the correct
>solution here.  The first is the simple design - I'm a picky bastard,
>and to me, design and cleanliness are pretty important.  ;-)  A clean
>implementation of a VFS is definitely preferred in that department.
>  
>
You are right here: The KIO->Gnome-VFS bridge is quite ugly from design. 
But it would be a first step. In the future - with a common main loop 
and after improving the Gnome-VFS client API with things KIO requires - 
KIO could perhaps call directly into the async Gnome-VFS API without 
going via the internal KIO-slave IPC.

What concerns writing another VFS i'm a bit scared that the result might 
look 90% like KIO or Gnome-VFS. A VFS is just a VFS.

>Second is the usefulness of a gnome-vfs/kioslaves bridge.  That only
>helps GNOME and KDE apps.  GnuStep, Rox, CLI apps, and so on *will not*
>depend on an architecture built on GNOME or KDE libraries like that.
>Even if there were no technical reason not to, there's political
>reasons.  Yay humanity.  ;-)
>
>  
>
The dependencies of Gnome-VFS are not that many (mainly orbit, glib, 
dbus). Probably you won't get that a lot cheaper.

try: ldd /usr/lib/gnome-vfs-daemon and: ldd /usr/lib/libgnomevfs-2.so.0

(Btw. I don't know what bonobo does in Gnome-VFS. Is it required for the 
core functionality?)

KIO is probably a little harder to separate from KDE - that's one of my 
arguments for choosing Gnome-VFS.

Cheers,
Norbert