A common VFS and a Common conf-system [Part II]

[Jamie McCracken]

Sean Middleditch wrote:
> On Thu, 2005-03-03 at 01:54 +0000, Jamie McCracken wrote:
> 
>>Sean Middleditch wrote:
> 
> 
>>>Not really the same kind of 'synchronous' - that refers to initiating an
>>>operation (like an upload) and waiting until it's complete.  That's
>>>application behavior.  You can code that behavior using an asynchronous
>>>API.  And in fact, you'd *want* to code it that way, since that way that
>>>use can hit Cancel or watch a progress meter or so on.
>>
>>Depends, sync is faster thats why some p2p apps use it (I cant remember 
>>which) and they have multithreaded clients for doing multiple 
>>downloads/uploads.
> 
> 
> Again, completely different meaning of synchronous.  Whether your API is
> synchronous or not, the exact same syscalls are used in an FTP transfer.
> 
> 
>>stat() is also a synchronous operation which if you performed over a 
>>slow NFS volume would block your main loop too unless its threaded. You 
>>cant assume all functions will be async in nature - a number are 
>>synchronous and will block especially on slower volumes so these may 
>>aggregate to cause bigger latencies.
> 
> 
> This is true.
> 
> However, stat() really isn't what this library is going to be making a
> whole lot of use of.  The whole reason for its existence is network
> servers, and *any* network protocol can be implemented asynchronously.
> Event stat() can be done asynchronously, really, although in a very
> non-portable fashion.  ;-)
> 
> I just don't feel that the potential for something like stat blocking
> for a few seconds is really that strong of a case for the far greater
> complexity that would occur by using only a threaded daemon.  Not to
> mention that is also then means you *must* force all file access through
> the daemon, even file: access (which is the one that would cause all the
> problems).
> 
> As I said in my last mail, if a particular backend finds that it has
> problems due to a few blocking syscalls or implementation problems, it
> can do the threads on its own.
> 
> i.e., the daemon says, "hey file backend, I need the meta-data on
> file:///usr/share/blah, let me know when it's done."  The file: backend
> knows that it could block, potentially for many many minutes, so it
> deals with starting and managing threads to carry out its labor.  The
> daemon never has to worry about it.  If the backend is run in-process
> with the app, it still doesn't have to worry about it.
> 
> I really don't feel even that will be needed, though.  Unless you have
> some specific examples of actual problems that occur?  Not just
> theoretical stuff.
> 
> I really don't want to over-engineer this system based purely on
> speculation.
> 
> If threads *are* necessary, I'm alright with that - but I want concrete
> proof that we need it first.
> 
> Given the stage in the game we're at, it's not even that big of a
> problem right now.  I'd rather we spend our time discussing "what does
> the VFS have to do" and not "how will it do it."  :)

Im not trying to shoot down your proposal - far from it! We all want and 
need something like this but it needs to be better than what we 
currently have (IE faster, easier to use etc - essentially something 
that just  blows away gnome-vfs completely rather than be a clone with 
different clothes).

I know threading makes things harder for you but it actaully makes the 
API friendlier and we can use sync without blocking  which would make 
things simpler for apps to use (the async part would be optional so apps 
can decide which to use as appropriate).

If you stuff everything in one big loop you either bear the brunt of 
potential blocks or pass the buck to the backends which will probably 
cause a lot of grief there. Maybe im being pessimistic and the problems 
I brought up wont affect ordinary users but I feel using threads gives 
added peace of mind to these potential pitfalls.

I dont have concrete proof of the need of threads because ive never used 
a threadless vfs.

jamie.










> 
>>jamie.
>>
>>
>>