protocol handling spec?

[Avery Pennarun]

On Tue, Aug 10, 2004 at 12:58:45AM +0100, Dave Cridland wrote:

> On Mon Aug  9 23:33:06 2004, Avery Pennarun wrote:
> >Sorry, KDE already implemented the bizarre URI handler you speak of at
> >least, it hands back the *listing* of the contents of your folder, which
> >is of course exactly what you want.  Then you can request (or not, as you
> >wish), each individual message separately.  And the URI handler itself
> >can handle caching.
>
> Are you telling me it hands you some weird XML formatted data which 
> lists every message, or it just tells you the number of messages? I 
> hope it's not going as far as a UID SEARCH 1:*, let alone a UID FETCH 
> 1:*. That would be really horribly slow. Evolution does the for my 
> own folders, when I run that, and it takes forever - I never let it 
> near the anonymous IMAP servers I sometimes use with several thousand 
> messages in.

I don't know what it does or how it retrieves the index; but my IMAP folder
is on a slow remote server with lots of messages, and KDE's imap: uri reads
that folder very fast.  Remember that the *implementation* of the imap: uri
can include caching, meaning that the *user* of the imap: uri doesn't have
to be smart in order to achieve good performance.

I can tell you for sure that it doesn't do anything stupid like download
every single message in my mailbox.  Of course not.  It downloads a list of
messages, and to get other messages I download... other URIs.

> >The API you proposed is nice and simple, but if it were me, I might want
> >to add a special function for specifically composing/decomposing lists. 
> >More than just imap: will want to create lists of objects that I can
> >parse and navigate in a general way.  Of course, HTML and XML are both
> >reasonable(*) ways of creating general types of lists.
>
> Erm. You're thinking that composing everything into XML is going to 
> be faster, somehow?

XML people like the idea that you could do it that way.  I, and apparently
you, don't particularly.  That's fine; encode it in any other format, or
even extend your API to understand the concept of a "folder listing".  It
can be as fast as you need, but more speed means less generality.

> >(BTW, a more general URI-type mechanism is the "moniker", which is simply
> >a mapping of strings into objects.  When you request a moniker for a
> >particular factory, it'll give you a corresponding object that implements
> >a particular API.  A URI is a particular kind of moniker that returns
> >objects implementing the URI API.)
> 
> What I think you think, in these terms, is that:
> 
> 1) One of:
> a) Each possible interface to an object returned by a moniker 
> corresponds to a MIME type.

If you think I think that, my powers of expression must be way worse than I
ever realized, and I apologize.  Where to begin?  Whew.

> b) All objects support an interface which provides an octet-stream, 
> and has a property of a MIME type. (Sans parameters, because nobody 
> likes them, eh?)

Umm, well, all typical URIs can provide objects which support such an
interface, yes, and for each example you've asked me about, I've explained
how that is possible.  Feel free to ask about more types of URIs; I can do
them all.

The parameters are merely optional, not required to be absent, although they
must be uniform across all URI types (since the interface must be uniform if
you want to be accomplishing anything).

> 2) All possible instances of all possible URIs return a uniform 
> interface. See above.

Certainly not.  There is *one* API which, for all supported URIs, must
provide an object instance that supports that API.  This is the API your
favourite browser apps, like Konqueror and Nautilus, might need.  When you
type a URI into your Nautilus or Mozilla URL bar, this is the API you're
*expecting* the objects to provide, or you wouldn't have typed the URI
there.

There are plenty of other potential APIs that might be supported by any
given subset of monikers, but if your calling app doesn't know about that
API, it doesn't help much.

> 3) There exists an interface for an octet-stream, with a MIME type 
> (with parameters, because they're useful for dispatch). This can be 
> passed to, and used by, an application without the application 
> needing to know about the specific method of access to the 
> octet-stream.

Agreed.

> 4) URIs may provide an interface as in (3), but may not. URIs do, 
> however, provide some interface with a single method which performs a 
> default action for them based purely on the scheme name.

Sure.  But the API of claim #4 is merely a subset of the API of claim #3; if
we say that every kind of URI has a single default method *that takes
optional input and output blobs*, then we can have it always (like in #4)
and flexible (like in #3).

> You seem to add in MIME types somewhere for no very good reason, 
> possibly using them as generalized interface names.

No, MIME types are separate from this, which is the point I'm trying to get
across; this discussion started out as an attempt to merge the MIME and URI
manager specs into a single spec, and I was pointing out that they're
different things with different uses, albeit similar-sounding ones.  I'm not
against merging the specs, only against misusing the actual concepts.

To use your terminology, if a URI can retrieve an octet-stream, that
octet-stream *does* have a MIME type, as all octet-streams do.  That
MIME-type is really a moniker that will give you an object (the
viewer/printer/editor/etc for that type of file), and that object is the
*user* of the URI itself.

> By the way, pop quiz, what should you do with a "dav" scheme URI? 
> (Hint, it's a registered one at IANA, and quite definitely doesn't 
> have a MIME type...)

This is an easy one: I handle it in almost the exact same way as I would
http (since it is, basically, http with a few extensions).  While dav itself
has no MIME type, the files retrieved or published via dav *do* have a MIME
type, and therefore I would load the application associated with that MIME
type, passing it the URI as the file to view/edit/print/etc.

> No, it works merely because I've told GNOME to fire off my email 
> client whenever it's asked to launch an imap scheme URI. That's dumb 
> handling - GNOME itself doesn't know anything about imap URIs other 
> than they start with {'i','m','a','p',':'}, so it can't know that 
> some IMAP URIs are accessible as an octet-stream, and others are not.

Okay, I agree that that's dumb.

> GNOME has all the functionality needed to launch arbitrary URIs. It 
> doesn't know, however, and I have no way of telling it, that some 
> URIs are reasonable to treat as filenames. (Or whichever your 
> preferred nomenclature is. Data addresses?)

Ah, so this is the heart of the problem.  We shouldn't be arguing about
whether URIs have MIME types; it's you that wants them to, not me.

Turn your URI into an object that implements the URI API.  Ask the URI API
to give you the blob.  Find the MIME type of the blob (if this is a common
operation, maybe "get mime type of retrieved object" should be part of the
URI API).  Run the preferred viewer for that MIME type.  Done.

> Worse, if I did tell GNOME, KDE wouldn't know. Nor Rox.

Indeed, this is a bad problem, and well worth solving.  I'm glad we're
getting to the serious business here.

> >The MIME type of an email message retrieved from any URI, including
> >imap:, should be message/rfc822.  When you display such a message, you
> >should expect it to pop up in some sort of "normal" mail reading program
> >or component, not just a text viewer.  Of course, you could also
> >configure your "default viewer" for that type of object to be a text
> >viewer, since it happens to be a text file.
>
> Reread the URI. It's pointing to the TEXT section of your message, 
> which was precisely the MIME type *you* set it to be, and I merely 
> quoted off the server. Not message/rfc822, text/plain; 
> charset="us-ascii".

Okay then; it should display in the default viewer for a text file.  A URI
that points into the actual body part of a message has nothing to do with
email, and therefore should not launch my mail reader, and it won't as long
as the imap: handler tells me the right MIME type.  Do you want it to?

On the other hand, if I point at a message/rfc822, I probably *would* expect
it to launch my mail reader (or at least an embedded component of it),
because whenever I view an email on my desktop, I expect it to look the
same.

> >This is why your IMAP client is the URI handler, and your mail reader is
> >the MIME handler.
> 
> I strongly suspect that would lead to a truly revoltingly slow mail 
> reader.

And yet it doesn't, and the KDE people proved it.  You should check your
assumptions.  I guess the most important thing to remember is that the IMAP
handler can do all the caching it wants, and just because you *ask* it for a
message doesn't mean it has to go and *retrieve* the message.

Have fun,

Avery