[Intel-gfx] [PATCH i-g-t 5/5] tests/kms_test_only: Validate TEST_ONLY correctness against full atomic commit

[Maarten Lankhorst]

Hey,

Op 12-01-17 om 11:28 schreef Mika Kahola:
> On Thu, 2017-01-12 at 11:11 +0100, Maarten Lankhorst wrote:
>> Op 30-12-16 om 13:00 schreef Mika Kahola:
>>> This test case adds TEST_ONLY flag to the following test cases to
>>> test
>>> atomic commit correctness.
>>>
>>>  - kms_plane_multiple
>>>  - kms_atomic_transitions
>>>  - kms_plane_scaling
>>>  - kms_rotation_crc
>>>
>>> The test randomly selects one of the above test cases and tests
>>> atomic
>>> commit. If the test fails with TEST_ONLY flag the real deal atomic
>>> commit
>>> is executed and the outcome is verified.
>>>
>>> The test runs by default for 64 iterations.
>>>
>>> Signed-off-by: Mika Kahola <mika.kahola at intel.com>
>>> ---
>>>  tests/Makefile.sources |   1 +
>>>  tests/kms_test_only.c  | 455
>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>  2 files changed, 456 insertions(+)
>>>  create mode 100644 tests/kms_test_only.c
>> Hey,
>>
>> What does this test add?
> This is for VIZ-6956. The idea here is to test atomic commits with
> TEST_ONLY first, and it that fails we try to commit in real. 
>> Any test should pass with PASS. This test retrying failed test-only
>> tests won't add much there. No test should ever fail.
> That's my thought also. You could argue whether or not this test adds
> any value. In my mind, this test only tests if TEST_ONLY flag is
> behaving ok :)
But you can already compare test passes vs test fails. There's no need for a separate test because any test failing should already be investigated.

I don't think a test that tests another test is useful, we already have infrastructure for that. :-)

~Maarten