test infrastructure ideas appreciated ...

[Michael Meeks]

Hi Kohei,

On Tue, 2015-06-09 at 21:00 -0400, Kohei Yoshida wrote:
> On Wed, 2015-06-03 at 14:33 +0100, Michael Meeks wrote:
> > Having said that writing unit tests is every
> > developers' responsibility where feasible
> 
> So, this "where feasible" is quite often subject to a wide range of
> interpretation.

	Right; so - I personally agree we need more tests =) Also that we need
to make it much easier to write new ones - so there is less excuse for
not doing it.

>   For some, writing a unit test is often not "feasible"
> if a bug fix happens during a busy release cycle because "there is no
> time to write a test" even when a fix was technically fully
> unit-testable (which BTW I fully disagree).

	So - AFAICS -every- bug-fix is potentially unit-testable - it is just a
matter of the cost to implement that: "stop world, spend months writing
large test infrastructure" etc. ;-) is possible just not really
feasible.

	My yard-stick would be that if writing the unit-test takes longer than
finding & fixing the bug - then we have a problem, and we're going to
struggle to encourage people to do that. No idea if that matches other
people's.

> I hope that we can also strive to change the culture amongst developers
> in this project and make a unit test a quasi-requirement for every bug

	Indeed :-) I think by far the best way to make that more socially
acceptable is by making it incrementally easier to write the next unit
test.

> But I believe what's equally important is to raise awareness on the
> importance of writing tests and make it as important as (if not more
> important) fixing the bug itself, and promote that mindset.

	Agreed; and I like the idea:

> If there is any way track bug fixes without corresponding unit tests and
> make that data available in some way, then that to me would be a very
> helpful tool to ensure that every bug fix comes with a test to prevent
> it from happening again.

	Markus had some interesting scripts for this I think. I'd also like to
prioritize regression bug fixes as needing unit tests - after all we
already messed that up once so it's prolly more likely than elsewhere to
have a future problem.

	I will add: "Produce good data on who did and didn't write unit tests
for fixes" to the infrastructure ideas list - social encouragement is
clearly important.

	Thanks !

		Michael.

-- 
 michael.meeks at collabora.com  <><, Pseudo Engineer, itinerant idiot