[Mesa-dev] [Bug 87136] Incorrect/undefined behavior from shifting an integer too far

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=87136

            Bug ID: 87136
           Summary: Incorrect/undefined behavior from shifting an integer
                    too far
           Product: Mesa
           Version: unspecified
          Hardware: Other
                OS: All
            Status: NEW
          Severity: normal
          Priority: medium
         Component: Mesa core
          Assignee: mesa-dev at lists.freedesktop.org
          Reporter: brucedawson at cygnus-software.com

While compiling Google Chrome with VC++'s /analyze (static code analysis) I
discovered a class of bugs in the mesa code base. There are three instances of
this bug where the integer '1' is shifted by an amount that ranges from zero to
47. '1' will have type 'int' and on basically every modern platform 'int' is 32
bits. Shifting an int more than 30 positions leads to undefined behavior
because the result will, necessarily be unrepresentable.

Even if we ignore the annoying spectre of undefined behavior, the behavior will
most definitely not be what is intended. On Intel processors the likely result
is going to be equivalent to shifting by shiftAmount&31 which means that 16
mask values will be repeated.

The warning is:
warning C6297: Arithmetic overflow:  32-bit value is shifted, then cast to
64-bit value.  Results might not be an expected value.

The fix is to use BITFIELD64_BIT(i) instead of (1 << I).

The locations where I have noticed this bug are:

drivers\dri\i965\brw_fs.cpp(2164):
   for (int i = 0; i < FRAG_ATTRIB_MAX; i++) {
      if (!(fp->Base.InputsRead & BITFIELD64_BIT(i)))
continue;

      if (prog->Name == 0)
         key.proj_attrib_mask |= 1 << i; // BUG

swrast\s_span.c(767):
         for (i = 0; i < FRAG_ATTRIB_MAX; i++) {
            if (span->interpMask & (1 << i)) {
               GLuint j;
               for (j = 0; j < 4; j++) {
                  span->attrStart[i][j] += leftClip * span->attrStepX[i][j];
               }
            }
         }

swrast\s_span.c(788):
         for (i = 0; i < FRAG_ATTRIB_MAX; i++) {
            if (span->arrayAttribs & (1 << i)) {
               /* shift array elements left by 'leftClip' */
               SHIFT_ARRAY(span->array->attribs[i], leftClip, n - leftClip);
            }
         }

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20141209/97940e97/attachment.html>