[Intel-gfx] Pixel-perfect frame checks in IGT Chamelium tests and CRC

[Paul Kocialkowski]

Hey,

On Fri, 2017-06-16 at 18:09 +0800, Chih-Chung Chang wrote:
> On Fri, Jun 16, 2017 at 5:28 PM, Paul Kocialkowski
> <paul.kocialkowski at linux.intel.com> wrote:
> > Hi,
> > 
> > On Thu, 2017-06-15 at 21:08 -0700, chihchung at google.com wrote:
> > > Paul Kocialkowski於 2017年6月15日星期四 UTC+8下午9時57分09秒寫道：
> > > > Hi,
> > > >
> > > > So far, there are two ways of testing for pixel-perfect frames using the
> > > > Chamelium that are in IGT. The first one grabs a full frame from the
> > > > Chamelium
> > > > and compares it pixel-to-pixel with the cairo reference, which works
> > well
> > > > for
> > > > DP/HDMI.
> > > >
> > > > For VGA, this is probably not the case (because the link is analogue).
> > In
> > > > that
> > > > case, I will look into implementing some fuzzy testing, probably
> > inspired
> > > > by
> > > > what piglit (probably) does to compare output frames with references.
> > > >
> > > > For pixel-perfect testing, grabbing a full frame and testing it with
> > memcmp
> > > > comes with a significant time penalty (about 2 seconds for 1080p). The
> > > > Chamelium
> > > > also provides a CRC mechanism that is faster and does not require
> > retrieving
> > > > the
> > > > frame, that IGT currently also supports. It compares the CRC calculated
> > by
> > > > the
> > > > Chamelium (implemented in the HDL) with a hardcoded reference value.
> > > >
> > > > This approach currently fails for me (the values I get don't match the
> > > > hardcoded
> > > > reference). There are reasons why it is not really reasonable: fonts
> > > > rendering
> > > > may change between machines (e.g. use of anti-aliasing) and cairo
> > version
> > > > changes could introduce slight rendering changes too (not to mention
> > changes
> > > > in
> > > > the test pattern itself). So instead of comparing the CRC with a
> > hardcoded
> > > > reference value, I think it would make a lot more sense to actually
> > > > calculate
> > > > the CRC based on the cairo image that is the actual reference (and that
> > we
> > > > should assume may change between runs/machines).
> > > >
> > > > I am currently looking into the CRC calculation mechanism used by the
> > > > Chamelium
> > > > and trying to reproduce it in C code. Is this a known algorithm for
> > which a
> > > > reference/optimized implementation exists, or something custom that the
> > > > folks
> > > > over at Google came up with?
> > > >
> > > > Any thoughts, comments or suggestions?
> > >
> > > I feel bad about the stupid hash algorithm I came up with, but here is the
> > > document:
> > > https://docs.google.com/document/d/1_HjEMA8fBoHkUbpUZq-OXtITfiomCb1HBKN07T
> > -POl
> > > Y/edit#heading=h.jqek3kkh9qjm
> > > You can also ask it to hash just part of the frame instead of the whole
> > frame
> > > (i.e. cropping before hashing).
> > 
> > Thanks for the link to the relevant documentation! I have been following it
> > closely and came up with the following implementation, which does not
> > produce
> > the same result. Would you have any idea of what I'm doing wrong?
> > 
> > That function is called with m = 4 and k = { 0, 1, 2, 3 }. I am using a
> > single-
> > color-frame and the 4 hash16 I get from the Chamelium are identical. They
> > are
> > however different from the 4 (also identical) that I get from :
> > 
> > int hash16(unsigned char *buffer, int width, int height, int k, int m)
> > {
> >         unsigned char r, g, b;
> >         long int sum = 0;
> >         int count = 0;
> >         int index;
> >         int hash;
> >         int value;
> >         int i;
> > 
> >         for (i=0; i < width * height; i++) {
> >                 if ((i % m) != k)
> >                         continue;
> > 
> >                 index = i * 3;
> > 
> >                 r = buffer[index + 0];
> >                 g = buffer[index + 1];
> >                 b = buffer[index + 2];
> > 
> >                 value = r | (g << 8) | (b << 16);
> >                 sum += ++count * value;
> 
> Maybe this multiplication overflows?

Good pick, that was it! Now the CRC is the same as the one obtained from the
Chamelium.

Thanks a lot!

> >         }
> > 
> >         hash = ((sum >> 48) ^ (sum >> 32) ^ (sum >> 16) ^ (sum >> 0)) &
> > 0xffff;
> > 
> >         return hash;
> > }
> > 
> > I am certain that the r, g, b values are correct.
> > 
> > Thanks!
> > 
> > --
> > Paul Kocialkowski <paul.kocialkowski at linux.intel.com>
> > Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
> > 
> > --
> > N.B. This list is shared with folks outside Google so please make sure no
> > confidential information is being discussed.
> > ---
> > You received this message because you are subscribed to a topic in the
> > Google Groups "chamelium-external" group.
> > To unsubscribe from this topic, visit https://groups.google.com/a/google.com
> > /d/topic/chamelium-external/ITCjBMJHfGM/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to chameliu
> > m-external+unsubscribe at google.com.
> > To post to this group, send email to chamelium-external at google.com.
> > To view this discussion on the web visit https://groups.google.com/a/google.
> > com/d/msgid/chamelium-external/1497605300.1451.9.camel%40linux.intel.com.
> 
> 
-- 
Paul Kocialkowski <paul.kocialkowski at linux.intel.com>
Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo