[Nouveau] [PATCH] nvidia/noveau: Fix color mask

[Michael Büsch]

On Wed, 17 Jun 2015 20:47:17 -0400
Ilia Mirkin <imirkin at alum.mit.edu> wrote:

> On Wed, Jun 17, 2015 at 1:05 PM, Michael Büsch <m at bues.ch> wrote:
> > The expression (~0 >> x) will always yield all-ones, because the right
> > shift is an arithmetic right shift that will always shift ones in.
> > Accordingly ~(~0 >> x) will always be zero.
> > Hence 'mask' will always be zero in this case.
> >
> > Fix this by forcing a logical right shift instead of an arithmetic
> > right shift by using an unsigned int constant.
> >
> > Signed-off-by: Michael Buesch <m at bues.ch>
> 
> Confirmed that this does indeed happen with
> 
> #include <stdio.h>
> int main(int argc, char *argv[]) {
>   unsigned mask = ~(~0 >> (32 - (argv[1][0] - '0')));
>   printf("%08x\n", mask);
> }
> 
> I guess fbdev/nvidia/nv_accel.c was the source of all this, as the
> code is identical, and it probably came first.


If anybody is able to help me in creating a working semantic patch
(coccinelle) for this, that would be great.

I found this using a very hacky and incorrect spatch (some
version is attached). It throws many false positives, doesn't find all
such bugs and does not create correct patch output (especially the
#define related rule is just meant as a hint).

Some basic thoughts that come to mind that could possibly be statically
checked somehow are:

- right shift of promoted variables. That is stuff like this:
  u8 x, y = 0x0F;
  x = ~y >> 1;
  /* x is 0xF8, not 0x78 as someone might expect. */
- Also check this for typedef'ed types where promotion takes place (that
  are smaller than int)?
- right shift of signed constants (like in this case). That probably is
  wrong in most cases.
  How to check signedness of constants in spatch? (123 vs 123U)
  Is that even possible?
- Also detect this stuff, if variables/constants are hidden via #define
  or such:
  #define REGVAL	0x0F
  writereg(REGISTER, ~REGVAL >> 1);

Probably more stuff could be checked. Ideas are welcome. :)

-- 
Michael



@@
u8 e;
expression s;
@@
- ~e >> s
+ (u8)~e >> s

@@
s8 e;
expression s;
@@
- ~e >> s
+ (s8)~e >> s

@@
u16 e;
expression s;
@@
- ~e >> s
+ (u16)~e >> s

@@
s16 e;
expression s;
@@
- ~e >> s
+ (s16)~e >> s

@@
char e;
expression s;
@@
- ~e >> s
+ (char)~e >> s

@@
unsigned char e;
expression s;
@@
- ~e >> s
+ (unsigned char)~e >> s

@@
short e;
expression s;
@@
- ~e >> s
+ (short)~e >> s

@@
unsigned short e;
expression s;
@@
- ~e >> s
+ (unsigned short)~e >> s




@@
constant c;
expression s;
@@
- ~c >> s
+ (unsigned int)~c >> s





@sh expression@
identifier val;
expression shift;
@@
val >> shift


@@
expression e;
identifier sh.val;
@@
- #define val ~e
+ #define val e

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/nouveau/attachments/20150618/aa8c5de0/attachment.sig>