[PATCH 1/1] video: fbdev: fix divide error in fbcon_switch

[Saeed Mirzamohammadi]

Attached the Syzkaller C reproducer.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: repro.c
Type: application/octet-stream
Size: 8761 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/dri-devel/attachments/20201021/7d7d070d/attachment-0001.obj>
-------------- next part --------------


> On Oct 21, 2020, at 4:57 PM, saeed.mirzamohammadi at oracle.com wrote:
> 
> From: Saeed Mirzamohammadi <saeed.mirzamohammadi at oracle.com>
> 
> This patch fixes the issue due to:
> 
> [   89.572883] divide_error: 0000 [#1] SMP KASAN PTI
> [   89.572897] CPU: 3 PID: 16083 Comm: repro Not tainted 5.9.0-rc7.20200930.rc1.allarch-19-g3e32d0d.syzk #5
> [   89.572902] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.5.1 01/01/2011
> [   89.572934] RIP: 0010:cirrusfb_check_var+0x84/0x1260
> 
> The error happens when the pixels value is calculated before performing the sanity checks on bits_per_pixel.
> A bits_per_pixel set to zero causes divide by zero error.
> 
> This patch moves the calculation after the sanity check.
> 
> Signed-off-by: Saeed Mirzamohammadi <saeed.mirzamohammadi at oracle.com>
> Tested-by: Saeed Mirzamohammadi <saeed.mirzamohammadi at oracle.com>
> ---
> drivers/video/fbdev/cirrusfb.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/video/fbdev/cirrusfb.c b/drivers/video/fbdev/cirrusfb.c
> index 15a9ee7cd734..a7749101b094 100644
> --- a/drivers/video/fbdev/cirrusfb.c
> +++ b/drivers/video/fbdev/cirrusfb.c
> @@ -531,7 +531,7 @@ static int cirrusfb_check_var(struct fb_var_screeninfo *var,
> {
> 	int yres;
> 	/* memory size in pixels */
> -	unsigned pixels = info->screen_size * 8 / var->bits_per_pixel;
> +	unsigned int pixels;
> 	struct cirrusfb_info *cinfo = info->par;
> 
> 	switch (var->bits_per_pixel) {
> @@ -573,6 +573,7 @@ static int cirrusfb_check_var(struct fb_var_screeninfo *var,
> 		return -EINVAL;
> 	}
> 
> +	pixels = info->screen_size * 8 / var->bits_per_pixel;
> 	if (var->xres_virtual < var->xres)
> 		var->xres_virtual = var->xres;
> 	/* use highest possible virtual resolution */
> -- 
> 2.27.0
>