[PATCH 9/9] tests: Add test to verify tests' ini files get loaded

[Pekka Paalanen]

On Fri, 3 Apr 2015 10:33:02 -0700
Bryce Harrington <bryce at osg.samsung.com> wrote:

> On Fri, Apr 03, 2015 at 03:21:45PM +0200, Marek Chalupa wrote:
> > On Fri, Apr 3, 2015 at 4:16 AM, Bryce Harrington <bryce at osg.samsung.com>
> > wrote:
> > 
> > > config-malformed uses an invalid configuration file to verify that the
> > > test harness is indeed attempting to load the test's requested config
> > > file.
> > > +
> > > +#include "config.h"
> > > +
> > > +#include "weston-test-client-helper.h"
> > > +
> > > +TEST(config_malformed_test)
> > > +{
> > > +       /* We should never reach here, because the invalid config file
> > > +        * should cause the server to exit with failure before invoking
> > > tests.
> > > +        */
> > > +       assert(1 == 1);
> > >
> > 
> > Maybe I don't understand what this test does, but assert(1 == 1) is always
> > true so this is kind of useless statement, isn't it?
> > Maybe you meant to use 1 != 1? And going further - couldn't be this fail
> > test converted to normal test, so that we are sure that the failure is
> > really
> > the one we test for?
> 
> Anything could be put there; the test itself never actually gets
> executed, because the failure occurs during the setup phase.  I'm not
> sure if XFAIL is the right approach here, it's just the first thing I
> found that reliably worked.

Hi,

since this is an XFAIL test... would it not signal a false success if
there was a bug and Weston tried to load a config file that does not
exist?

Specifying a config file that does not exist is intended to cause an
error-exit too.

> We also need a 'positive test', that is - set some parameter in the .ini
> file, then have the test verify the setting is being set.  I'd love to
> do this with the mode parameter, but as I found this seems to not be
> recognized by the headless backend.  A few other parameters I looked at
> could be set via .ini, but I didn't see how to query from the
> client-side.  So, I leave all that for future work, but I think being
> able to verify set parameters is pretty important.

I see two possible ways:

1) Convert the test into a plugin. Plugins can access the config read
by Weston.

2) Check WESTON_CONFIG_FILE environment variable:
http://cgit.freedesktop.org/wayland/weston/commit/?id=6c71aaeec5d034e052c9d95f3c36b5b38069f6d8
Could even load the config via that, and check set values, in case one
is testing generated configs.


Thanks,
pq