test infrastructure ideas appreciated ... -- automatically generated unit tests

[Zolnai Tamás]

2015-06-16 9:15 GMT+02:00 Stephan Bergmann <sbergman at redhat.com>:
> On 06/15/2015 03:51 PM, Zolnai Tamás wrote:
>>
>> I've got an idea too: automatically generated unit tests.
>> I think with a clang plugin we can generate unit tests for all
>> class/struct in the source code in build time. Build system can link
>> it as a simple unit test and run it directly after generation. If it
>> returns with no error it can be removed. If it returns an error the
>> test should be untouched to allow running it directly (without
>> regeneration).
>>
>> These kind of tests gets in my mind now:
>> - Create an object and remove it (check no problem with destruction)
>> - Create and remove an object thousand of times (memory leak)
>> - Copy operator and copy constructor result is equivalent (operator== is
>> needed)
>> - Copy an object, remove the original object (shallow copy)
>> - Create a const object, call all const method on it and check it remains
>> equal.
>> - Getter and setter method: set a value and check the corresponding
>> getter method returns with the same value
>> -- Set values depending on the parameter type
>> --- int: end points of the value range, negative value, 0, positive value
>> --- string: empty string, dummy string, string with wild characters.
>
>
> I'm not sure how exciting an approach that is.  Especially for the later
> part of your list, it builds on the assumption that an object is merely a
> conglomeration of public member functions, that can be called in arbitrary
> sequence and with arbitrary arguments, and that is often just not the case.
> The dreaded JunitTest_*_unoapi are modeled somewhat like that.

Yeap, that's right. There are cases when they are not arbitrary, but
when that's the case I think it's always a good idea to add assertions
(or warnings or comments or anything) to the code to make assumptions
explicit about the arguments or order of the member functions. The
unit test generator should use these information.

Best Regards,
Tamás