Why Coroutines?

[Lawrence D'Oliveiro]

Every GUI toolkit defines its own event-loop architecture for
performing concurrent tasks while remaining responsive to asynchronous
events. The Python developers no doubt saw that there were so many
event-loop implementations to choose from, so let’s create another one
<https://docs.python.org/3/library/asyncio.html>!

What’s different about Python’s asyncio architecture is its integrated
support for coroutines
<https://docs.python.org/3/reference/compound_stmts.html#coroutines>, a
new language feature introduced in Python 3.5. Coroutines offer a more
natural way of implementing a task that requires a number of
potentially time-consuming steps to be performed in sequence without
blocking the event loop, as opposed to the usual technique of chaining
long sequences of callbacks (or resorting to threads, with all the
synchronization problems they can cause).

I don’t think the Python developers intend for asyncio to replace the
event loops provided by existing GUI toolkits. Instead, it should be
possible to provide adaptors for those event loops so they behave like
subclasses of asyncio.AbstractEventLoop. This way, Python code can be
written to be event-loop-agnostic--perhaps the first time this has been
achieved in any language.

I have been taking full advantage of this coroutine capability in DBussy
<https://gitlab.com/ldo/dbussy>, <https://github.com/ldo/dbussy>, my
pure-Python wrapper around libdbus.

For example, consider the “introspect_all” example script: given a bus
name, it introspects the entire object tree defined by that service,
recursively expanding any <node> tags that it finds. The basic logic of
the introspect_path routine looks like this:

    async def introspect_path(path) :
        result = await «result of introspecting path»
        for i, node in enumerate(result.nodes) :
            expanded = await introspect_path(path + [node.name])
            «replace result.nodes[i] with expanded»
        #end for
        return \
            result
    #end introspect_path

As you can see, it is a recursive function coded in a fairly obvious
way--no big surprise in the logic, at any rate. The only thing special
is that it is defined as an “async” (coroutine) function, which means
it is called via the “await” construct. Note also the use of another
“await” to make the actual introspecting D-Bus call: this is what gives
the event loop a chance to keep running while waiting for the reply
message to come back.

Just for fun, there is another coroutine, called “show_progress”,
running concurrently with this one, that provides a running display of
the state of the introspection calls. For example, introspecting
the bus name “org.freedesktop.systemd1” on the system bus on my machine
takes about 4 seconds and finds over 300 separate object paths.  The
output is about 14,000 lines; it would be longer, but my script replaces
duplicated interface definitions with stubs so you only see their
details once.