[Intel-gfx] Pixel-perfect frame checks in IGT Chamelium tests and CRC

[Chih-Chung Chang]

On Fri, Jun 16, 2017 at 5:28 PM, Paul Kocialkowski <
paul.kocialkowski at linux.intel.com> wrote:

> Hi,
>
> On Thu, 2017-06-15 at 21:08 -0700, chihchung at google.com wrote:
> > Paul Kocialkowski於 2017年6月15日星期四 UTC+8下午9時57分09秒寫道：
> > > Hi,
> > >
> > > So far, there are two ways of testing for pixel-perfect frames using
> the
> > > Chamelium that are in IGT. The first one grabs a full frame from the
> > > Chamelium
> > > and compares it pixel-to-pixel with the cairo reference, which works
> well
> > > for
> > > DP/HDMI.
> > >
> > > For VGA, this is probably not the case (because the link is analogue).
> In
> > > that
> > > case, I will look into implementing some fuzzy testing, probably
> inspired
> > > by
> > > what piglit (probably) does to compare output frames with references.
> > >
> > > For pixel-perfect testing, grabbing a full frame and testing it with
> memcmp
> > > comes with a significant time penalty (about 2 seconds for 1080p). The
> > > Chamelium
> > > also provides a CRC mechanism that is faster and does not require
> retrieving
> > > the
> > > frame, that IGT currently also supports. It compares the CRC
> calculated by
> > > the
> > > Chamelium (implemented in the HDL) with a hardcoded reference value.
> > >
> > > This approach currently fails for me (the values I get don't match the
> > > hardcoded
> > > reference). There are reasons why it is not really reasonable: fonts
> > > rendering
> > > may change between machines (e.g. use of anti-aliasing) and cairo
> version
> > > changes could introduce slight rendering changes too (not to mention
> changes
> > > in
> > > the test pattern itself). So instead of comparing the CRC with a
> hardcoded
> > > reference value, I think it would make a lot more sense to actually
> > > calculate
> > > the CRC based on the cairo image that is the actual reference (and
> that we
> > > should assume may change between runs/machines).
> > >
> > > I am currently looking into the CRC calculation mechanism used by the
> > > Chamelium
> > > and trying to reproduce it in C code. Is this a known algorithm for
> which a
> > > reference/optimized implementation exists, or something custom that the
> > > folks
> > > over at Google came up with?
> > >
> > > Any thoughts, comments or suggestions?
> >
> > I feel bad about the stupid hash algorithm I came up with, but here is
> the
> > document:
> > https://docs.google.com/document/d/1_HjEMA8fBoHkUbpUZq-
> OXtITfiomCb1HBKN07T-POl
> > Y/edit#heading=h.jqek3kkh9qjm
> > You can also ask it to hash just part of the frame instead of the whole
> frame
> > (i.e. cropping before hashing).
>
> Thanks for the link to the relevant documentation! I have been following it
> closely and came up with the following implementation, which does not
> produce
> the same result. Would you have any idea of what I'm doing wrong?
>
> That function is called with m = 4 and k = { 0, 1, 2, 3 }. I am using a
> single-
> color-frame and the 4 hash16 I get from the Chamelium are identical. They
> are
> however different from the 4 (also identical) that I get from :
>
> int hash16(unsigned char *buffer, int width, int height, int k, int m)
> {
>         unsigned char r, g, b;
>         long int sum = 0;
>         int count = 0;
>         int index;
>         int hash;
>         int value;
>         int i;
>
>         for (i=0; i < width * height; i++) {
>                 if ((i % m) != k)
>                         continue;
>
>                 index = i * 3;
>
>                 r = buffer[index + 0];
>                 g = buffer[index + 1];
>                 b = buffer[index + 2];
>
>                 value = r | (g << 8) | (b << 16);
>                 sum += ++count * value;
>

Maybe this multiplication overflows?


>         }
>
>         hash = ((sum >> 48) ^ (sum >> 32) ^ (sum >> 16) ^ (sum >> 0)) &
> 0xffff;
>
>         return hash;
> }
>
> I am certain that the r, g, b values are correct.
>
> Thanks!
>
> --
> Paul Kocialkowski <paul.kocialkowski at linux.intel.com>
> Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
>
> --
> N.B. This list is shared with folks outside Google so please make sure no
> confidential information is being discussed.
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "chamelium-external" group.
> To unsubscribe from this topic, visit https://groups.google.com/a/
> google.com/d/topic/chamelium-external/ITCjBMJHfGM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> chamelium-external+unsubscribe at google.com.
> To post to this group, send email to chamelium-external at google.com.
> To view this discussion on the web visit https://groups.google.com/a/
> google.com/d/msgid/chamelium-external/1497605300.1451.9.
> camel%40linux.intel.com.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/intel-gfx/attachments/20170616/a5d22743/attachment-0001.html>