VIO - Visual IO Agents

[nf2]

hi, maybe you will find this interesting/worth discussing. i'm currently 
working
on a prototype of:

VIO - Visual IO Agents

(a possible successor of KIO Slaves and Gnome-VFS modules)

one idea for VIO is to standardize the interface between clients
and vfs-protocol handlers (agents) by using a tiny point 2 point
socket communication library written in plain C that protocol
handlers can be used in a cross-desktop way.

the second idea is that access to remote resources can be provided
by ordinary gui-based applications. (something
similar old fashioned gftp, winscp, file-roller, kftp...), but with
a stripped down gui, most of the time resting as (blinking)
icon in the systray.

you would just launch such a client application, connect to a server
or open a zip-file and the resource would become accessible to *all*
applications and filemanagers through a local socket in the tmp dir:

/tmp/vio-{username}/ftp%user%ftp.xyz.org

thus, there would probably be no need for standardizing login dialogs,
password storage interfaces, remembering locations etc...
the user would have full control on
connections, he can monitor them, cancel them etc...

mounting a resource means launching an agent, by pressing the
agents "close" butten the resource is unmounted. there could be a
"preferred protocol handler" config table somewhere to enable
auto -launching.

it doesn't matter if the user runs the kde, gnome or whatever
implementation of such a "protocol - provider",... the development
of protocol-handlers would become completely decoupled from
desktop development...

the design of libvio would be quite simple -
just scanning the /tmp/vio-{username}/ directory for sockets and
getting file/directory access through a
simple binary point to point socket protocol...

currently i see no need for a centralized "vio-daemon".


excerpt of the documentation:

VIO Agents
----------

Vio agents provide acess to certain virtual file-system
resources through a socket protocol. They can run as seperate process
or inside the client as threads.

There are four types of VIO-Agents

1) SHARED
   have a GUI and accept multiple connections through a
   local socket in a specified directory:
   /tmp/vio-{username}/ftp_user at ftp.xyz.org  
   they handle protocols like ftp:, smb:, zip:...
   they usually run as one instance per resource
   (like user at ftp.xyz.org) and act as a kind of proxy.
  
   you can think of them as stripped down versions of
   applications like gftp, kftp, file-roller,...
   they might be minimized to the systray and only
   pop up for authentication, etc... 
   the user can monitor activity, control them, close them
   with their the GUI.
  
2) PRIVATE
   are launched by the client application and usually don't
   have a GUI. they handle local resources like file:, computer:
   usually they are single threaded like KIO slaves.
   communication goes through a pair of pipes - usually stdin/stdout
   of the agent process.
  
3) PRIVATE-AS-THREAD
   same as 2) but running inside the client as threads, using
   a socket-pair for communication.

4) SPECIAL
   like 2) or 3) but handle special tasks like file-copying,
   monitoring, launching etc...
  
  
VIO Library (libvio)
--------------------

both sides - the clients and agents link to the small C-library libvio.so.

libvio provides a low level API to launch and communicate with VIO agents
and to run vio-jobs. the interface is message-loop style
on the client-side and KIO-SlaveBase style on the agent side.
on the client side it is designed in a way that various nicer high-level
synchronous and asynchronous APIs (like KIO::Jobs, Gnome-VFS and
future g-vfs can be wrapped around.

public interface to be used on the client side:
   vio.h, vio_file.h, vio_codes.h
public interface to be used on the agent side:
   vio_agent.h, vio.h, vio_file.h, vio_codes.h


VIO Job types:
--------------

get      : read entire file
put      : write entire file
file     : open/read/write/seek file
ls       : read directory
copy     : copy file
move     : move file
stat     : get filinfo
charttrs : change file attributes
mkdir    : create directory
rmdir    : remove directory
delete   : remove file


VIO Socket Protocol
-------------------

The transport mechanism in vio is
message-container based.
the wire format of the messages looks like this:

[ 16 byte header ; body: byte array ]

typedef struct {
    int magicNr;
    int sequenceNr;  /* sequence number (job id) */
    int typeID;        /* message type */
    int bodySize;
} VioTransMsgHeader;

a connection can only handle one job at a time, der sequenceNr is
only used to throw away messages from previous jobs (like cancel
messages which arrived too late)

concurrent jobs will trigger launching another agent (for
private agents) or opening a second connection to a shared
agent.

the internal tranport machanism is defined in vio_trans.h. 
Structure type messages (like directory entries) often hold their
data in a VioTransStruct - a dictionary type (like KIO:UDSEntry)
which can be serialized. see vio_trans_struct.h.

D-Bus will not be used for the core tasks of VIO, but might be used
for file/dir change notification and for emitting agent
startup/shutdown events. not sure yet.


VIO Messages
------------

VioMessage is a kind of variant data type, which has the following fields
- depending on the message type. VioMessage is the data-type visible
to the library user. access to the fields is provided
through getter and setter functions. you should never access the
internal structs directly.

pseudo definition of message types and fields:


COMMAND_* Message               # invoke a job
[
    location : viofile
    (targetLocation) : viofile   # for copy/move operations
    commandOptions : enum
]

/* agent redirection happens when an agent cannot handle a task
himself. he might delegate the job to a child agent for instance. */

REDIRECT_TO_AGENT Message
[
    specialAgent : enum      # for instance the "file copy agent" / 
otherwise 0
    socketName/Port  : string/int
]

PROGRESS Message
[
    total  : size
    count  : size
    validFields  : bit[]
    unit  : enum             # unit of the "total", "count" fields
                             # (bytes, percent,...) 
    stage  : enum            # authentication, waiting, connecting, 
downloading, uploading...
]

DATA_CHUNK Message
[
    data  : byte[]
    size  : size
]

FILE_INFO Message
[
    location : viofile
    name : string
    user : string
    group : string
    hasPermissions : bool
    permissions : bit[]
    size : size
    iconname : string
    hasMimetype : bool
    mimetype : string
]

CANCEL Message
[
]

ERROR Message
[
    code : enum
    message : string
]

DONE/FAILED Message
[
    returnValue : enum   
    willLiveUntil : date        # date when the connection will be 
become invalid
                                # and might be closed.
                                # connection should not be reused after 
this "date"
]

...

Message-Communication Schemes
-----------------------------

___ GET Job ___

[client]   ...   [agent]

+===== CMD_GET ========>
<---- REDIR2AGENT------+
<----- PROGRESS(*) ----+
<===== DATA_CHUNK =====+
<===== DATA_CHUNK =====+
<===== DATA_CHUNK =====+
...
-------- CANCEL ------->
<------- ERROR --------+
<===== DONE/FAILED ====+

___ PUT Job ___


[client]   ...   [agent]

+===== CMD_PUT ========>
<---- REDIR2AGENT------+
<----- PROGRESS(*) ----+
+===== DATA_CHUNK =====>
+===== DATA_CHUNK =====>
+===== DATA_CHUNK =====>
...
-------- CANCEL ------->
<------- ERROR --------+
<===== DONE/FAILED ====+


___ COPY Job ___

[client]   ...   [agent]

+===== CMD_COPY =======>
<---- REDIR2AGENT------+
<----- PROGRESS(*) ----+
-------- CANCEL ------->
...
<------- ERROR --------+
<===== DONE/FAILED ====+

MOVE Job

see COPY Job


___ LS Job ___

[client]   ...   [agent]

+====== CMD_LS ========>
<---- REDIR2AGENT------+
<----- PROGRESS(*) ----+
<===== FILE_INFO =======
<===== FILE_INFO =======
<===== FILE_INFO =======
...
-------- CANCEL ------->
<------- ERROR --------+
<===== DONE/FAILED ====+



___ STAT Job ___

[client]   ...   [agent]

+===== CMD_STAT =======>
<---- REDIR2AGENT------+
<----- PROGRESS(*) ----+
<===== FILE_INFO =======
-------- CANCEL ------->
...
<------- ERROR --------+
<===== DONE/FAILED ====+



___ CHATTRS, MKDIR, RMDIR, DELETE Jobs ___

[client]   ...   [agent]

+===== CMD_COPY =======>
<---- REDIR2AGENT------+
<----- PROGRESS(*) ----+
-------- CANCEL ------->
...
<------- ERROR --------+
<===== DONE/FAILED ====+


___ FILE Job ___

[client]   ...   [agent]

+===== CMD_FILE =======>
<---- REDIR2AGENT------+
<----- PROGRESS(*) ----+

+===== READ N =========>
<===== DATA_CHUNK =====+
...or...
+===== DATA_CHUNK =====>
<===== WRITE_ACK ======+
...or...
+====== SEEK ==========>
<====== SEEK_ACK ======+
...
-------- CANCEL ------->
...
<------- ERROR --------+
+====== CLOSE =========>
<===== DONE/FAILED ====+


...

regards,
norbert