system and desktop VFS merged

[Sean Middleditch]

On Sun, 2005-03-27 at 22:32 +0300, rosen georgiev wrote:
> Sean Middleditch wrote:
> 
> > On Sun, 2005-03-27 at 17:01 +0300, rosen georgiev wrote:
> > 
> > > So there should be :
> > > 
> > > a project to build the vfs layer, something like a set of librarys (eg
> > > libvfsftp, libvfsftp, libvfsssh and so on) that, as sed manage the
> > > connection reuse, passwords, security ... (this is something i'm not sure
> > > can be done - is it possible to make eg. libvfsftp so that connect will
> > > return the same connection for a set of processes, if they don't explicitly
> > > ask for a new one?)
> > 
> > You must use a daemon, which we will be using no matter what, as it is
> > simply impossible to provide the desirable semantics without one. The
> > daemon can do all the magic of connection sharing, password querying and
> > caching, file caching, whatever we need.
> 
> sorry, i didn't want to go there again :)
> isn't it possible to make the libs a interface to the daemon (if not - skip the 
> next 7 lines)

That's the idea.  :)

> What i was thinking is that there should be a lib for every protocol used
> giving almost the same functionality + the shareing of connections, 
> passwords and ..., it should not be POSIX. Then D-VFS will use that to

No, no reason to have separate libs.  Accessing different protocols
should be 100% transparent to both developers and users.  There is a
single library used by applications and developers.  That library is
just a convenience wrapper around the protocol to the daemon.  The
daemon then has loadable backends for various protocols.

> implement its high level API, but there could also be other apps or APIs
> that will use this "vfs layer" to give POSIX access - they are not part of 
> D-VFS (may be there will be no such apps) and they will still share the 
> connection.

I recommend that legacy apps wanting to use POSIX use something like
FUSE with a D-VFS backend.  It would also be possible to do a
pure-userspace "almost POSIX" library - something like gnome-vfs -
though I see no good reason for an application to port to such a library
when they could port to D-VFS itself, especially since I want to make
the D-VFS API much easier and less error-prone than POSIX.

(As a side note, I expect almost nobody to actually use the real D-VFS
API.  I expect people to use a wrapper based on Qt, glib, etc.  The
D-VFS API should be designed for easy wrapping and skip all convenience
functions, and let higher-level wrappers and toolkits make the API super
pleasant.  This is actually rather essential to getting good behaviour
with, for example, synchronous API calls - you need to let the
application process toolkit events while waiting for the operation to
complete.)

> 
> > > 
> > > and something to replace FUSE-KIO so that it uses the vfs layer and if 
> > > possible cross patform (HURD will also be easy - but not very used :)
> > 
> > Adding a FUSE layer to work on top of D-VFS would be quite possible and
> > something I fully expect to happen.
> > 
> 
> Yes, but isn't it better (in means of speed) to have a FUSE fs that talks to 
> the protocol it implements directly (throught the vfs layer) and not 
> throught D-VFS

I doubt it.  The vast, vast majority of performance problems will be in
the network latency and bandwidth, not in the local code.  I would
honestly be surprised if you could even measure the difference in
performance between the two FUSE backend methods.

That said, it still isn't possible to do what you're asking without
using the daemon anyway.  The daemon is mandatory for connection sharing
and such.

> 
> > 
> > > 
> > > 
> > > It is obvious that this is not a complete cross platform solution but it
> > > _can_ be if BSD and others make something like FUSE for there kernel and it
> > > _does_ provide a complete Desktop VFS, the rest can be added in time
> 
> i didn't say that right, here is anothe try:
> D-VFS will be crossplatfrom, but the connectivity with non-desktop apps 
> will be available only on Linux throught FUSE-DVFS or something similar.

Yes, that sounds right.  Sorry for misinterpreting.

> 
> > 
> > First, and I've said this more times than I ever cared for, FUSE is NOT
> > THE ANSWER, and NEVER WILL BE. FUSE is 100% limited to working with
> > POSIX, and POSIX only, and cannot offer functionality in addition to
> > POSIX, and that isn't acceptable, because POSIX simply CANNOT do certain
> > things we want. Simple things like "copy this file" cannot be done with
> > POSIX - you must instead read/stream the file from the source to the
> > destination - on a remote share, that means in order to copy a file
> > (which could otherwise be done 100% on the server-end) you must download
> > the entire file and then upload it back, just to make a copy. POSIX
> > isn't up to snuff, FUSE can only offer POSIX, and thus FUSE is not the
> > answer. It's a great hack to make legacy apps work somewhat well with
> > modern file system operations, but it is not and never, ever should be
> > the basis of a modern VFS layer.
> > 
> > I fully support a FUSE module for D-VFS, but I will not see D-VFS get
> > hamstrung by being designed entirely around FUSE and its limitations.
> > 
> 
> i wasn't saying that FUSE is the answer, but it could be the answer for the
> legacy apps that Desktop apps call (the IDE/gcc case). FUSE things are
> not part of D-VFS and are not mandatory on a system.

Right.

> 
> if the vfs layer thing is not possible , what about the first 2 of the 
> Desktop VFS "standarts" - they are just rules and are very easy to implement,
> and totaly transparent to the API, _and_ will make it really easy for everyone
> who wants to use the likes of FUSE-DVFS. I'will paste them here again
> 
> 1) 	The API of the VFS must except filenames in absolute or URI
> 	form. both protocol://<path> and /vfs/protocol/<path> should 
> 	be valid, where "/vfs" must be a standart for all APIs. It must be
> 	totaly transparent to the program

I really see no purpose in using the VFS path stuff.  Classic UNIX uses
mounting.  If a user wants their legacy app to access a remote share,
they should mount it just like they currently do with sshfs or smbfs or
whatever.  They can mount it anywhere they want - I usually mount remote
shares in ~/.mount/ for example.  If a user wants a dynamic mount area,
I'd suggest having the individual users mount such a dynamic filesystem
in their home folder as well.

If people are going to implement system-level hacks to access D-VFS, I
really do think the absolute best bet is to make open() recognize a URI
and start talking to the D-VFS daemon instead of the kernel.  Better
performance, better compatibility with other D-VFS apps, and easier to
port to other systems.  There are some problems with such an approach,
but they're things I'm quite sure the Free Software community can
tackle.  You're all a bright bunch.  ;-)

> 2)	All programs that use a Desktop VFS and spawn processes
> 	must give them the absolute path (/vfst/protocol/<path>). except
> 	for the file:// protocol in this case file://<path> will be just <path>
> 
> The files under /vfs  will be owned by the user who accessed them
> and that is save enough right?

I don't think there's any real security problem with a /vfs system.
Just general ugliness.  ;-)

> 
> 
> still eager :)
> 
> rosen
> 
> -----------------------------------------------------------------
> http://gbg.bg/search - Изпробвайте още сега най-добрата българска търсачка!
> _______________________________________________
> xdg mailing list
> xdg at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xdg
> 
-- 
Sean Middleditch <elanthis at awesomeplay.com>