"make test" target (was Re: [Portland] xdg-utils proof of concept)

[Bryce Harrington]

On Tue, May 02, 2006 at 07:42:36AM -0500, Jeremy White wrote:
> > Anyway, please give 'make test' a shot and let me know if anyone runs
> > into strange issues.  If this looks like an ok approach to folks, I'll
> > expand the tests to cover the rest of the scripts, as I have time.
> > It would be great if when people add new functionality to a given
> > script, if they could also update it's test file to include test(s) for
> > that functionality.
> 
> Hey Bryce,
> 
> I spent some time looking at this.
> 
> I think that real testing is going to require
> interaction with the user.  (For example, I just
> tried to make a menu - did it actually appear?)
> 
> On the other hand, xdg-copy fails quite simply
> for me, and manual tests would pick up on that kind of
> failure handily.
> 
> So I think we should also add another mechanism; maybe
>   INTERACTIVE=1 make test
> ?

Yeah, that could work.  I'm definitely more interested in
non-interactive tests that can be automated, but you're probably right
that the most interesting results will require interactive testing.

Plus, I'm always a fan of, "Automate the easy stuff, and leave the hard
stuff manual, until it gets annoying enough that your brain finds a
trick to automate that too."  ;-)

> I'm working up an interactive xdg-utils test program.
> In it's current form it's fairly silly, but I intend to
> break it out into pieces suitable for using as tests.
> 
> Is there a convention for doing interactive tests
> during a 'make test' run?

I haven't seen examples, but then most of the testing I've done has
been for non-GUI stuff.  Using an environment variable to turn it on
seems sensible to me.

Bryce