[systemd-devel] Ubuntu, Upstart, and systemd

[Albert Strasheim]

Hello

On Wed, Apr 25, 2012 at 6:47 AM, David Strauss <david at davidstrauss.net> wrote:
> Mark Shuttleworth posted recently on Ubuntu sticking with Upstart [1]:
> I have a few questions I'm hoping people here can help with:
> * How accurate is the claim that Upstart has far more comprehensive
> automated testing? If systemd significantly lags here or could -- in any
> case -- stand improvement in automated testing, are there any plans in the
> works to remedy that?

I agree with this criticism to some extent. I don't know if some of
the developers have some kind of test suite outside of the Git
repository?

I'm guessing Lennart and others do a lot of ad-hoc testing using
systemd-nspawn and/or containers though.

I think systemd needs two kinds of tests.

Firstly, some unit tests for some of the basic functions in src/core.
This code looks pretty good (this assertion being supported by the
fact that systemd works in the real world), so this isn't necessarily
highest priority.

Secondly, system tests built as sets of .service/.socket/etc. files
and "fake" services that tests every sane combination of options
documented in the systemd.* manual pages. This should probably be
combined with a quick way of booting any one of these setups with
systemd-nspawn or in a QEMU/KVM virtual machine and a test driver
program that can perform a bunch of actions on the running image:
start a service, stop a service, kill a process, check that systemd
state is sane after each action, check for errors in logs, etc.

We've been using systemd extensively (lots of services, lots of
dependencies) for about a year now with great success, but we've found
2 or 3 bugs which have been difficult to report because we haven't
been able to make them reproducable without our internal services and
their configurations.

For example, we've found cases where systemctl restart causes systemd
to not pass in the sockets to a socket-activated service. The fake
service/test driver approach detailed above could easily check for
this kind of thing.

Having a set of these tests in place could also help others to quickly
create a reproducable test case for a problem they see in their
system.

If no-one beats us to it, we're planning to build and release a first
step towards this system test suite described above in the next few
months.

Regards

Albert