[Piglit] Rework profiles

[Dylan Baker]

On Tuesday, January 27, 2015 15:25:54 Ilia Mirkin wrote:
> Picking a random group:
> 
> # Group ARB_shader_objects
> with profile.group_manager(
>         PiglitGLTest,
>         grouptools.join('spec', 'ARB_shader_objects')) as g:
>     g(['arb_shader_objects-getuniform'], 'getuniform', run_concurrent=True)
>     g(['arb_shader_objects-bindattriblocation-scratch-name'],
>       'bindattriblocation-scratch-name', run_concurrent=True)
>     g(['arb_shader_objects-getactiveuniform-beginend'],
>       'getactiveuniform-beginend', run_concurrent=True)
>     g(['arb_shader_objects-getuniformlocation-array-of-struct-of-array'],
>       'getuniformlocation-array-of-struct-of-array', run_concurrent=True)
>     g(['arb_shader_objects-clear-with-deleted'],
>       'clear-with-deleted', run_concurrent=True)
>     g(['arb_shader_objects-delete-repeat'], 'delete-repeat',
>       run_concurrent=True)
> 
> So, to start, I really like this approach. (IIRC I even mentioned at
> one point that I wanted to do something like this, but time has
> been... lacking.)
> 
> However there's further redunancies. e.g. every test starts with
> arb_shader_objects-, and takes no args.

I do like the idea of being able to pass a prefix into the group_manager
call.

> Secondly, every test in this
> group is concurrent. Would be nice to encode that into the group
> manager, IMO, so that each line would look like

I had planned to follow on with a couple of patches switching the
concurrent flag, to make run_concurrent=True the default, and False the
special case, which I think would accomplish that level of simplicity
for most cases.

> 
> g('getuniform')
> 
> And everything else would be implied. Thoughts?
> 
> [Obviously you need to retain the configurability to do crazy things
> for the few groups that require it.]
> 
> On Tue, Jan 27, 2015 at 2:58 PM, Dylan Baker <baker.dylan.c at gmail.com> wrote:
> > This is a very large, very invasive rework of the way that test profiles
> > work, hopefully for the better. The goal was to create a simple syntax
> > that discouraged variation, and limited accidents. What I ended up with
> > is a two fold change in the way tests are added.
> >
> > First, shader tests, glslparser tests, and asmparser tests are
> > automatically added to the profile with *zero* modification to all.py (A
> > similar setup could be used by cl.py, but I wanted the input of the cl
> > devs, since it would mean either renaming or moving tests). These tests
> > use their directory structure to determine what groups they belong to.
> >
> > This necessitated another change, treating all group and test names as
> > case insensitive. This will also fix the problem on case sensitive file
> > systems of having tests split across groups in the html summary.
> >
> > The second change it the removal of add_plain_test and
> > add_concurrent_test, they are replaced by a method in TestProfile that
> > gives an 'adder' function, which encompasses all the power of
> > add_concurrent_test and add_plain_test, but also has all of the power of
> > adding tests directly to the profile.
> >
> > The third change is the removal of TestProfile.tests, the nested
> > dictionary structure which was flattened before running, and replacing
> > it with the use of TestProfile.test_list, a dictionary of string:Test
> > values that assumes that the tests are already formatted appropriately,
> > this series of changes makes extensive use of the grouptools module to
> > ensure future flexibility in this regard.
> >
> > Patches 1-2 are out as a separate series.
> > Patches 3-5 are groundwork changes
> > Patch 6 adds two new profiles that I have carried around for some time
> > and are very useful for development
> > Patches 7-30 Are the bulk of the series.
> >
> > Patches 16-18 will need to be merged into one mega-patch to allow clean
> > bisect. I have tried to come up with a better way to do this, but can't
> > come up with any other solution that allows bisecting all portions of
> > piglit except the mega patch
> >
> > The end result, even with additional testing and two new results
> > versions (which add a lot of code for updating and validating), is ~+500
> > LOC, even with my efforts to wrap lines better in all.py then they were
> > previously.
> >
> > This is available at my github:
> > https://github.com/dcbaker/piglit rfc/flat-test-list
> >
> > _______________________________________________
> > Piglit mailing list
> > Piglit at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/piglit
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20150127/5b739b49/attachment.sig>