[Mesa-dev] [PATCH 0/3] RadeonSI GLSL "lit" tests

[Marek Olšák]

On Tue, Aug 2, 2016 at 9:23 PM, Nicolai Hähnle <nhaehnle at gmail.com> wrote:
>
>
> On 02.08.2016 21:06, Marek Olšák wrote:
>>
>> On Tue, Aug 2, 2016 at 3:40 PM, Nicolai Hähnle <nhaehnle at gmail.com> wrote:
>>>
>>> On 31.07.2016 22:29, Marek Olšák wrote:
>>>>
>>>>
>>>> Inspired by LLVM, I created a small test suite that looks like "lit"
>>>> tests, except that the input is GLSL and it verifies asm.
>>>>
>>>> The motivation was to verify that we generate optimal code when starting
>>>> from GLSL.
>>>>
>>>> [PATCH 1/3] radeonsi: add environment variable SI_FORCE_FAMILY
>>>> [PATCH 2/3] radeonsi: add a standalone compiler amdgcn_glslc
>>>> [PATCH 3/3] radeonsi: add GLSL lit tests
>>>
>>>
>>>
>>> Good idea. Two comments:
>>>
>>> Please use .glsl or some such instead of .ll as a file extension. Those
>>> files are not LLVM IR.
>>>
>>> There's a `make check` in Mesa, which I readily admit to not using
>>> enough.
>>> There are r300_compiler_tests that are hooked up to it. Perhaps it's
>>> worth
>>> wiring that up?
>>
>>
>> Some of the tests currently fail and will continue to fail for quite
>> some time. I don't think people would appreciate it if "make check"
>> always failed.
>
>
> llvm-lit has an "expected failures" mode which could be used. It works by
> adding a line
>
> ; XFAIL: *
>
> to the test.

"expected failures" doesn't say whether the failures are good or bad.
I've always understood "expected failures" as "failing is the correct
behavior here".

Also, I guess people would be very unhappy that their make check
failures can only be fixed by building LLVM from source, or worse
(better?), by writing patches for LLVM. ;)

Marek