better unit tests

[Stefan Sauer]

hi,

first, thanks to everyone contributing to our unit tests. In this,
somewhat lengthy, mail I'd like to share a few point how I believe we
can improve our tests.


The main advice - keep tests small.

A test needs a name. If you test tons of things in a single test, it is
hard to name it. A good name helps to see what is broken when the test
fails.
A broken test needs to be debugged. A small test is easy to debug, since
you don't need to step through unrelated things.
A test needs to be isolated. When a test grows long, chances are that
things start to get coupled.


Follow the AAA rule - Arrange Act Assert

Ideally each test has 3 parts:
- Arrange: setup stuff, like create objects and configure them
- Act: invoke the api you want to test
- Assert: a single assert verifying that the api did what it was
supposed to do

The advantage of this pattern is, that it makes the naming easy and
helps to keep the tests short.


Don't repeat yourself.

This one is controversial. The idea is that you don't assert things you
already tested elsewhere. E.g. you make one test for
gst_element_factory_make() and assert that it works. Other tests that
use gst_element_factory_make() in the arrange phase, *do not assert*. If
gst_element_factory_make() get broken, we will get one test failing with
an assertion and a bunch of crashing tests. You will debug the test that
has the assertion failure. Since this will be names e.g.
"test_element_factory_make_creates_element", this failing means this
feature is broken. If we assert in other tests e.g.
test_send_custom_events, then such a assertion failure is non obvious.


Add test helpers.

Extract common code into small static helpers. Should be a no-brainer
and helps to get an idea about what the test is doing.


I found this book quite nice, don't worry that this is about c#, it is
quite generic.

https://www.amazon.de/Art-Unit-Testing-Roy-Osherove/dp/1617290890

In any case you'll find this also in various online blogs.


Stefan