continued: Common-VFS proposal

[nf2]

Hi,

Havoc Pennington wrote:

>On Tue, 2005-01-25 at 15:38 +0100, nf2 wrote:
>  
>
>>Havoc Pennington wrote:
>>
>>    
>>
>>>gnome-vfs has one huge design flaw, which is that it looks like
>>>the POSIX file API. It should instead be a very very simple 
>>>core interface ("get entire document", "put entire document",
>>>"list documents") 
>>> 
>>>
>>>      
>>>
>>That's not a design flaw - that's one of the examples where the 
>>Gnome-VFS design is correct (IMO).
>>
>>A shared backend in KIO style (async get/put only) won't work. The 
>>reason is that with async get/put you give up "flow-control".
>>    
>>
>
>NFS has *horrible* semantics for GUI apps (things just lock up, with no
>progress indication, no way to cancel, and no way to know how much work
>is left or what's happening).
>  
>
Maybe my previous statement was not clear enough. Of course GUI 
applications should always use an async interface (like the very 
friendly KIO:: or KIO::NetAccess[1] API which keeps the mainloop 
iterating). No matter if they access a slow external resource or read a 
3 GB file from the local harddisk.

I was talking about the design of a shared backend and the internal 
interface for VFS-modules. IMO a synchronous Interface is more 
"universal" here, because you can convert it to async in the frontend 
part of the VFS (like Gnome-VFS does it) but you don't loose the 
possibility to give certain applications more fine-grained synchronous 
(streaming) access. [2]

[1] KIO::NetAccess is not really a synchronous interface in terms of a 
traditional in-process open - give me more - close streaming interface 
(with flow-control). It just calls the main loop recursively until a 
complete File operation (download,..) has been finished by an async 
slave process and then returns from the function call. That's probably 
not what people who ask for a streaming interface want.

[2] Whats the point of having a Gnome-VFS style synchronous streaming 
interface?

Let's say you want to create a background converter which streams a file 
from one VFS location to another VFS location. In Gnome-VFS you just 
create a worker thread which opens two streams and loops through 
read()->convert Data->write().

In KIO you probably run crazy with two TranferJobs and (ring)buffering 
data inbetween receiving data() and answering dataReq() callbacks. You 
have to be really cautious that your buffer doesn't grow to much - in 
cases the destination is slower than the source. You will need all kind 
of magic with TransferJob::suspend()/resume() to get this right. And the 
performance will be terrible, because two slave processes and your 
main-loop are involved.


>Also, you can't *really* have POSIX semantics with the backends like
>http etc.; because they involve multiple system calls and generally lots
>more work with lots fewer guarantees than the POSIX APIs make.
>  
>
With "POSIX style" i meant a synchronous (streaming) interface because i 
thought thats what you you were critisizing in Gnome-VFS. Of course i 
didn't mean POSIX *specifically*.

>There are also lots of backends that simply don't support parts of the
>POSIX APIs, and it's much nicer to be able to query this in advance
>instead of an ENOSYS style approach.
>  
>
Definitely. I like your queryInterface proposal. The question is which 
"methods" of the vfs-interface are part of the standard API and which 
optional and protocol specific (like KIO::special). IMO all those 
required by file-management have to be "standard". Filemanagers will 
never call things like HTTP POST directly.

Such a queryinterface could be easily built into Gnome-VFS (together 
with specialAtomic, specialOpen, specialDo, specialClose methods maybe).

>
>There's no value at all to being POSIX-like, because 1) mounting a
>userspace VFS in the kernel is a really bad idea (see above about
>semantic guarantees) and 2) the POSIX API is completely inconvenient and
>doesn't do what most GUI apps want to do.
>  
>
The frontend API sould always stay as convenient as KIO:: and 
KIO::NetAccess. But there are special cases where a synchronous 
streaming interface is very usful IMHO.

>Visible threads in the API are bad, because neither GTK+ nor KDE are
>very thread-friendly. gnome-vfs has threads behind the scenes only.
>  
>
I meant threads behind the scenes which "post" callbacks as events to 
the main-loop thread. Just like in Gnome-VFS.

Cheers,

Norbert