[packagekit] spawned backends, and dispatcher

[Richard Hughes]

Tim Lauridsen and I got hacking one Friday night, the conversation went
kind of like this (summarised):

timlau: I've committed a change to the yum backend to use a dispatcher
hughsie: what's that?
timlau: Allows you to submit extra stdin to an existing python process
hughsie: cool, I'll do the C bits

Anyway, after a couple of hours of hacking we've converted the yum
backend to use a dispatcher. This means we don't have to keep reloading
it every time we click on something in the GUI, and everything is a lot
quicker. There's quite a bit of complexity about handling different
locales and proxy settings from different concurrent sessions, but it
all's unit tested and now seems to work pretty well.

I'll paste what I added to the docs:

There are two modes of operation for a spawned backend, a one shot mode
and a dispatcher mode. 

      * The "one-shot" mode is where commands are executed (e.g.
        search-name.py none power), which then initialises the package
        backend, returns results and then exits. This is obviously not
        ideal, as some backends like smart can take several hundred ms
        to initialise, and this leads to a "laggy" GUI applications. 
        
        This however is the easiest mode to implement, and the original
        one supported by all versions of PackageKit. 
        
      * The "dispatcher" mode is where a command is used to startup the
        backend, for instance yumBackend.py search-name none power and
        then the backend then sits and waits for more standard input.
        Further operations can be done on the loaded backend sending
        commands to stdin, e.g. search-name none power. If there are no
        more operations after a preset time (default 5 seconds) then the
        backend is sent exit over stdin, and the backend terminates. The
        daemon will ensure the operations are serialised, and that
        backends not sending finished are cleaned up properly. 
        
        This is quite easy to implement (see the end of yumBackend.py),
        and is supported from PackageKit version 0.3.2. The dispatcher
        mode does not have to implemented in python; any language that
        can read from stdin can block and be used in this way. 

Now, the dispatcher mode is only lightly tested, but seems to pass all
the self checks in the code with flying colours.

If you want to convert your code (looking at you, afb and kenvandine)
then it's really simple. Just change your calls in your
pk-backend-xxxx.c from:

-       pk_backend_spawn_helper (spawn, "fooBackend.py", "get-packages.py", filters_text, NULL);
+       pk_backend_spawn_helper (spawn, "fooBackend.py", "get-packages", filters_text, NULL);

And then add something like this in fooBackend.py (and remove the tiny
*.py helper scripts):

def main():
    backend = PackageKitFooBackend('',lock=True)
    args = sys.argv[1:]
    backend.dispatch_command(args[0],args[1:])
    while True:
        line = raw_input('')
        if line == 'exit':
            break
        args = line.split(' ')
        backend.dispatch_command(args[0],args[1:])

if __name__ == "__main__":
    main()

I'm going to test the yum backend some more today with the dispatcher
mode turned on, and hopefully release 0.3.2 with that change on by
default.

Note, this isn't designed to stick around for a long time like the dbus
backends (maximum of 5 seconds), and so don't think for a minute the
dbus backends are deprecated. I will remove the yum2 backend, as it's
obvious that the yum backend is kept more up to date and is much more
stable.

Richard.