[Bug 109350] [CI][SHARDS] igt at kms_cursor_crc@cursor-alpha-(opaque|transparent) - fail - Failed assertion: !mismatch

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=109350

--- Comment #3 from Martin Peres <martin.peres at free.fr> ---
(In reply to Juha-Pekka Heikkilä from comment #2)
> These failing tests were introduced at
> 
> commit fde4dce431bf324939a982017169214e0fa00d4f
> Author: Mamta Shukla <mamtashukla555 at gmail.com>
> Date: Thu Jan 10 03:02:39 2019 +0530
> 
> 
> tests/kms_cursor_crc: Add test to check extreme alpha values for cursor plane
>  
>  Add test to check extreme alpha values i.e. fully opaque and fully
> transparent
>  for cursor plane and verify by calculating hardware and software CRC.
>  
>  Signed-off-by: Mamta Shukla <mamtashukla555 at gmail.com>
>  Reviewed-by: Harry Wentland <harry.wentland at amd.com>
> 
> 
> I don't find any log saying these tests ever passed on any machine,
> including report at patchwork https://patchwork.freedesktop.org/series/54465/

Did you assign Harry to make sure it passes at least on AMD HW?

Looking at the tests' code, I think that there might be some internal
conversions happening inside Intel's HW that leads to a slightly-different
output than what cairo is doing. We should talk to Ville about this to see if
we can mitigate the issue, or find a new way to implement the test while
keeping the intent.

In any case, this is a little dodgy to assign failures happening on Intel HW to
AMD engineers without justification other than "they added the added the new
test". IGT is not Intel-specific anymore and we definitely want other companies
to contribute to it by writing more generic tests (KMS) or HW-specific ones.

Maybe we should have collaborated more when the test was added, to allow some
intel engineers to review the failure result ahead of the merge. Most of the
failure is on me, because the report said "Success" and it does not make it
easy to see the regression (https://patchwork.freedesktop.org/series/54465/).
Here is the relevant part:

#### Suppressed ####

  The following results come from untrusted machines, tests, or statuses.
  They do not affect the overall result.

  * {igt at kms_cursor_crc@cursor-alpha-opaque}:
    - shard-kbl:          NOTRUN -> FAIL +1
    - shard-apl:          NOTRUN -> FAIL +1
    - shard-snb:          NOTRUN -> FAIL +1

  * {igt at kms_cursor_crc@cursor-alpha-transparent}:
    - shard-hsw:          NOTRUN -> FAIL +1
    - shard-glk:          NOTRUN -> FAIL +1

This is because new tests are automatically considered as suppressed because we
do not know how stable they are. However, we discussed today how we want to
make the results of new tests stand out in the report (MISSING -> $WHATEVER).
I'll be working on this with Tomi Sarvela during the week, since we should be
mostly ready to enable this.

Let's wait for Harry to confirm the tests at least pass on their HW, and we can
try to figure out why we get a difference between the SW and HW CRC (to reuse
the test's terminology).

This bug is quite a nice case :) Exciting times!

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/intel-gfx-bugs/attachments/20190204/53254b29/attachment.html>