virtual filesystem ideas

[Alexander Larsson]

On Mon, 2003-09-22 at 22:21, Ken Deeter wrote:
> > 
> > So, can you give an example when the observer would be used?
> >
> 
> Well, the main distinction between when to and when not to would just be
> whether you wanted to know what was going on, or whether you didnt. As
> you mentioned below, when the app does want to know about what is going
> on, it will have a progress bar or a throbber or something like that.
> The backend observer interface would be specifically used to monitor
> this kind of thing. I think it would be interesting if a program that
> knows its using http over SSL for example, could show you more
> information in a progressbar than just a percentage. The observer lets
> you capture that in a backend specific way, so we don't have to abstract
> the sense of "progress" among all back ends. I think usually when apps
> care whats going on at the lower level, then they want more than "10%
> done, 20% done, etc..", and when apps don't care, then they shouldn't
> have to.

This is very unlike the sort of API that gnome-vfs has though. In
Gnome-vfs all file operations are client-initiated. So you'd basically
call "read" when you want to read a buffer, instead of the vfs
downloading the whole file for you and sending you buffers when it gets
them. gnome-vfs does have some highlevel operations that could use such
a system though (the gnome_vfs_xfer calls), but that just uses the lower
level code, so such code might not really have a place in a vfs proper.

> As for the main vfs observer, the primary use i was thinking of was like
> a FAM type of application. If we are going to extend our filesystem reach
> to different kinds of fs's, then the alteration monitoring will surely be
> different across each backend. In this case you want a central way to
> capture what is going on, so that, say if you have konqueror looking
> at a ssh:// it might be able to pop up new files as they are created.

Sure, virtualized file change monitoring is required, so you have to
have some sort of callbacks.

> > In the ideal world, apps would use the vfs for all file access. Not all
> > file access is of the form "user asks the app to load a file". Much of
> > it is things that happen automatically, in the background, or as small
> > parts of everyday program work. When loading an icon for display you
> > can't have the vfs automatically poping up dialogs for you. And in the
> > case you want to have status while loading, chances are that you don't
> > want a standard dialog, but instead want to use some specific UI feature
> > of your app (throbber, status bar in window, etc).
> >
> 
> I agree with you. However, if the app hasn't implemented a progress bar
> or throbber, or has not explicity decided that nothing should be said,
> should it just say nothing then? I think this is the traditional
> assumption or I/O programming to this date, and in the GUI context I
> question it somewhat. I'm not saying there shouldn't be a choice of
> whether somthing does or does not pop up, but if we want to see what is
> going on, we should be able to. Maybe pop-up dialogs isn't the best way
> to do it, maybe you could have a panel applet that lists current ongoing
> vfs transactions or something. But even for like a command line program,
> I think it would be easier on the user if he had some visual indication
> that something was going on, w/o nec requiring every programmer to have to
> program that in. Besides most progress bars look kind of the same anyways,
> it is a huge duplication of effort.
>
> If we clearly define an interface in which a status bar type thing
> interacts with the backend, then the client API could even provide a standard
> "vfs status widget" that did the right thing.

My guess that setting up such an interface implementation would be about
the same amount of work as updating the status bar or progress bar. And
I don't think people will want to add such a standardized status widget
to their UI (which they painstakenly designed). I might be wrong though.
 
> > It is worse with dbus. For the case above the kernel switches are not
> > context switches. With dbus everything will go through the dbus daemon,
> > so you'll get: app -> dbus daemon -> vfs server -> dbus daemon -> app.
> > That is 4 context switches as opposed to the 2 that are "needed" for
> > out-of-process communication.
> 
> Well, i guess such is the price you pay for being able to abstract away
> whether you are using glib or qt main loops and such. Again, since I
> can't really predict the performance, and I'm not sure even where the
> bottlenecks will be, Just saying there are more context switches seems
> unconvincing to me. What if we are talking about loading http:// or smb://
> or something, you're bottleneck then is not likely to be context switches..
> you're process is gonna get switched out when its waiting for the network
> anyways.

Using dbus is not the only way to integrate with both glib and qt. dbus
is not the magical one-and-only solve-all-technology. 

You say "What if we are talking about loading http:/", I say "What if we
are talking about file://" (which will be the mostly used backed).
 
> > Sure, it is nice if there is some common code for doing some thing. But
> > there are so many different ways an error can be handled. Sometimes the
> > app expects an operation to fail, for instance when searching for a file
> > through a set of paths by calling open in each path. And even when a
> > failure is really a failure its almost always not interesting what the
> > lowlevel error was, but instead the i/o error needs to be propagated a
> > few levels up in the application where it can be either handled by doing
> > something different of a dialog with a much higher level description can
> > be presented to the user. 
> >
> 
> Right, so what I'm saying is if you don't care about what the error was,
> then let the system handle it in a standard way, which it can provide
> nice localized GUI messages or whatever. If you do want to know what the
> error was, then you specify an observer, and you can react exactly how
> yo want to.

So, you'll get a nice translated popup that says. "Not on the same file
system", "Operation not permitted", "File exists", "Looping links
encountered", or any other standard error message. The user gets totally
confused about what this means because of the lack of context (the vfs
doesn't know the context) and the app ignores the error, thinking the
standard dialog handles it and continues on, possibly causing further
problems down the line.

Experience has shown, time after time, that the only place you can
generate meaningful user error descriptions is at a higher level in the
application. And experience also shows that its almost always a bug to
ignore error, and it just leads to buggy software.

I know that handling this is more work. But the truth is that writing
good software *is* more work. Handling errors is *important*, and we
don't want to further the attitude that its not by letting apps ignore
them easily.
 
> > Yeah, maybe I was harsh on you due to this, but I just think the dbus
> > direction is wrong, and will cause us problems, while not solving really
> > many issues.
> >
> 
> Well whether it will cause us problems I don't know ;-) Whether it will solve
> issues, it seems like a pretty flexible way to me.. ;-)

But we really have no idea about the properties of dbus. Its not even
finished yet.
 
> > The gnome-vfs backends are in-process, but I'm moving some of them
> > out-of-process, since then cached data, authentication and connections
> > can be shared, and the backends get a much more well-specified
> > environment to run in. I think a common vfs system has to be out-of-proc
> > to have any chance to work, but that may mean bad performance for local
> > files, slowing down things like the file manager...
> >
> 
> konqueror seems to do ok for me ;-)

I think kio has a out-of-process bypass for local files i think.
 
> > > how uri's look: since there even seems to be RFC's regarding this issue,
> > > it would probably be wise to stick to the protocol:// type URI's
> > 
> > Thats not enough though. Both kde and gnome have extensions to the
> > availible RFCs.
> > 
> 
> Ok. But I guess my argument would be that if you do have a common vfs then your
> URI problems sort of go away because to the vfs, there is only one uri format.
> Whether this agrees with the current gnome/kde implementations... maybe not,
> but its probably a small matter of translation. I think its like saying 'well,
> how do you specify filenames on linux?', and the answer is 'in this way, because
> thats the only way it works'

Of course when you have only one implementation the way to do it is the
way that implementation does it. That does not start describe how we
*want* it to work. This needs design work, we can't just say it'll be
decided when the code is finished.
 
> > I don't care much about how the stacking is implemented, but how is
> > chaining exposed. gnome-vfs uses '#' to chain uris, libferris treats
> > e.g. zips as directories.
> >
> 
> Oh ok, well I guess this is almost a UI issue. I think there are advantages to both.
> Having it like a directory might make it easier for a user in the sense that
> he/she doesn't have to understand the whole stacking concept, but it might confuse
> them because there seems to be less distinction between a file and a directory.
> 
> Having explicit # is much more, well, exposed, but does the user really care? I tend
> to think what will matter in the end, is whether the zip file has a 'file' like icon
> or a 'folder' like icon in the file manager.

I don't know which one is best, or if there are other ways. My point is
that we need to think about these things. These are the things that
matter. (And its not only about the user visible UI, also about the
programmer visible behaviour.) 

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
                   alexl at redhat.com    alla at lysator.liu.se 
He's a jaded soccer-playing cowboy who knows the secret of the alien invasion. 
She's a scantily clad junkie queen of the dead from Mars. They fight crime!