[Intel-gfx] Pixel-perfect frame checks in IGT Chamelium tests and CRC

[Lyude Paul]

JFYI: Hardcoded CRCs are fine I'm pretty sure, but I might be wrong. As
well, put the chamelium in a metal box. The way the IO board is hooked
up is not really the way something running DP should be hooked up, so
it's very susceptible to electromagnetic interference. This will
usually cause CRC mismatches with DP.

This being said however, I think we should have some better functions
for the chamelium for doing frame comparisons, mainly something that
does fuzzy frame comparisons for stuff like VGA.

On Thu, 2017-06-15 at 16:57 +0300, Paul Kocialkowski wrote:
> Hi,
> 
> So far, there are two ways of testing for pixel-perfect frames using
> the
> Chamelium that are in IGT. The first one grabs a full frame from the
> Chamelium
> and compares it pixel-to-pixel with the cairo reference, which works
> well for
> DP/HDMI.
> 
> For VGA, this is probably not the case (because the link is
> analogue). In that
> case, I will look into implementing some fuzzy testing, probably
> inspired by
> what piglit (probably) does to compare output frames with references.
> 
> For pixel-perfect testing, grabbing a full frame and testing it with
> memcmp
> comes with a significant time penalty (about 2 seconds for 1080p).
> The Chamelium
> also provides a CRC mechanism that is faster and does not require
> retrieving the
> frame, that IGT currently also supports. It compares the CRC
> calculated by the
> Chamelium (implemented in the HDL) with a hardcoded reference value.
> 
> This approach currently fails for me (the values I get don't match
> the hardcoded
> reference). There are reasons why it is not really reasonable: fonts
> rendering
> may change between machines (e.g. use of anti-aliasing) and cairo
> version
> changes could introduce slight rendering changes too (not to mention
> changes in
> the test pattern itself). So instead of comparing the CRC with a
> hardcoded
> reference value, I think it would make a lot more sense to actually
> calculate
> the CRC based on the cairo image that is the actual reference (and
> that we
> should assume may change between runs/machines).
> 
> I am currently looking into the CRC calculation mechanism used by the
> Chamelium
> and trying to reproduce it in C code. Is this a known algorithm for
> which a
> reference/optimized implementation exists, or something custom that
> the folks
> over at Google came up with?
> 
> Any thoughts, comments or suggestions?
> 
> Cheers!
>