[Piglit] Ideas for Wine Test Suite and Mesa Workflow

[Zhenbo Li]

Hello, Emil

Thank you for your reply.

2016-02-19 23:10 GMT+08:00 Emil Velikov <emil.l.velikov at gmail.com>:
>>
>>> How would one detect between the states depends on you/wine devs.
>>> Having distinction between separate tests (i.e. one can run as
>>> many tests, independently, as possible) and tests and subtests
>>> would also be nice to have.
>> This is where things get tricky. Wine's d3d tests are in huge files that
>> contain lots of independent tests, but because they're in one file
>> (and thus one make target) you cannot run them independently. If you
>> simply use make's return value you cannot evaluate them independently
>> either.
>>
> If subtests are interdependent, then splitting/running them
> independently does not make much sense.
> Some reporting "subtest X passed/failed/etc" would be great through.
> Not too sure how the runner interprets/handles those currently.

In theory, the subtests[1] should be independent. However, in a few
dlls' test cases, all the subtests are heavily coupled.

If we stop the attempts on splitting the subtests(I think it's possible),
write a script to interpret the result, and give such a report would
not be difficult.

I think I can write a functional python script, and submit it to wine.
In mesa, it needs to do
little(more simple than piglit/tests/igt.py). I'm writing a proof of
concept test(piglit/tests/wine)
now, and I'm going to send it to this mail list few days later.

Thank you.

[1]: For example, run `wine d3d10_crosstest.exe --list`, one can get
Valid test names:
    device
    effect

And here are 10 subtests in effect: constant_buffer_stride
constant_buffer_type get_variable_by local_shader state_group_defaults
state_groups variable_element variable_member variable_type
variable_type_class



-- 
Have a nice day!
Zhenbo Li