dvfs api and toolkits

[Sean Middleditch]

On Tue, 2005-04-05 at 02:33 +0100, Jamie McCracken wrote:

>  From your wiki you criticised POSIX and its reading of chunks of bytes 
> so I assumed D-VFS would not have this :

Ugh, those are poorly worded.  My complete apologies.  I'll update them
to make my intentions a little clearer.  The gist was _supposed_ to be:
POSIX is based on opening a file and grabbing any chunk of bytes at any
time, where D-VFS is more intended for opening a file and grabbing the
whole thing - just not all in one big feel swoop.  ;-)

The point was more relevant to writing than reading.  In POSIX you can
do things like open a file, seek to byte 10, write out 12 bytes, jump to
byte 40, and truncate.  D-VFS is more intended for one of two cases:
write out a whole file (atomically if possible) or append data to a file
(atomically if possible).

Writing is actually where POSIX is the biggest pain for the average
developer, since far too many developers don't know/remember that
write() can do a partial write of the given data.  Lots of people get
read() wrong too, but there's not a whole ton D-VFS can do there without
in-depth knowledge of the file format being read.

Where POSIX focuses entirely on the concept of writing some bytes to a
section of a file, D-VFS is to concentrate on writing documents, and try
to get that as easy as possible while also making sure things like
atomicity or automatic backups are very easy and reliable (when
possible).

> > 
> > The APIs work similar to POSIX in that you get chunks of the file
> > streamed to you as you read.  With the async API it's delivered in
> > callbacks while with the sync API its read in a loop.  Either way, you
> > get a chunk, process it, and can then throw it out if you don't need it
> > anymore.  
> 
> Okay that wasn't clear before - I was under the impression the load 
> method loaded the entire file into memory without forcing the client app 
> to read chunks of bytes (as you criticised that in your wiki!).

Sorry, entirely my fault for bad wording.  This is why peer review is
good.  ;-)

> 
> 
> Playing a streaming video should work pretty flawlessly with
> > D-VFS.  Jumping around in a video might be a challenge.
> > 
> > The API will probably support seeking in files, but I want to highly
> > discourage developers to use that feature unless absolutely necessary,
> > because many protocols don't support it at all.
> 
> I think seeking is needed for other things in documents - PDF's perhaps?

And this is why I'm looking for input from application authors and not
discussion on daemons and protocols.  ;-)  *Do* PDF readers do seeking
around in files, or read the whole things into memory?  Do they want
seeking?  What about all the very useful file systems that *don't* offer
seeking support, like FTP - these apps should work on those file
systems, and caching is something I'd REALLY want to avoid if at all
possible - if it's something only one or two apps really need, I'd
rather just make those apps themselves do the caching, for example.

> 
> Glad to see you are considering supporting this.

Trying to consider just about everything.  That's why I'm still looking
for input instead of coding.  ;-)

> 
> jamie.
> 
> 
> > 
> > 
> >>jamie.
> >>
> >>
> 
> 
-- 
Sean Middleditch <elanthis at awesomeplay.com>