[Mesa-dev] [PATCH 0/3] RadeonSI GLSL "lit" tests

[Nicolai Hähnle]

On 03.08.2016 01:28, Marek Olšák wrote:
> On Tue, Aug 2, 2016 at 9:23 PM, Nicolai Hähnle <nhaehnle at gmail.com> wrote:
>>
>>
>> On 02.08.2016 21:06, Marek Olšák wrote:
>>>
>>> On Tue, Aug 2, 2016 at 3:40 PM, Nicolai Hähnle <nhaehnle at gmail.com> wrote:
>>>>
>>>> On 31.07.2016 22:29, Marek Olšák wrote:
>>>>>
>>>>>
>>>>> Inspired by LLVM, I created a small test suite that looks like "lit"
>>>>> tests, except that the input is GLSL and it verifies asm.
>>>>>
>>>>> The motivation was to verify that we generate optimal code when starting
>>>>> from GLSL.
>>>>>
>>>>> [PATCH 1/3] radeonsi: add environment variable SI_FORCE_FAMILY
>>>>> [PATCH 2/3] radeonsi: add a standalone compiler amdgcn_glslc
>>>>> [PATCH 3/3] radeonsi: add GLSL lit tests
>>>>
>>>>
>>>>
>>>> Good idea. Two comments:
>>>>
>>>> Please use .glsl or some such instead of .ll as a file extension. Those
>>>> files are not LLVM IR.
>>>>
>>>> There's a `make check` in Mesa, which I readily admit to not using
>>>> enough.
>>>> There are r300_compiler_tests that are hooked up to it. Perhaps it's
>>>> worth
>>>> wiring that up?
>>>
>>>
>>> Some of the tests currently fail and will continue to fail for quite
>>> some time. I don't think people would appreciate it if "make check"
>>> always failed.
>>
>>
>> llvm-lit has an "expected failures" mode which could be used. It works by
>> adding a line
>>
>> ; XFAIL: *
>>
>> to the test.
>
> "expected failures" doesn't say whether the failures are good or bad.
> I've always understood "expected failures" as "failing is the correct
> behavior here".

I think those cases are better expressed with `not` and the like.

> Also, I guess people would be very unhappy that their make check
> failures can only be fixed by building LLVM from source, or worse
> (better?), by writing patches for LLVM. ;)

That's a fair point. I don't feel that strongly about the `make check` 
integration anyway. My R-b also applies if you only change those file 
extensions.

Cheers,
Nicolai

>
> Marek
>