[PATCH V2] drm/modes: Fix divide error in drm_mode_debug_printmodeline

[Ville Syrjälä]

On Mon, Nov 20, 2023 at 10:41:18PM +0800, Edward Adam Davis wrote:
> [Syz Log]
> divide error: 0000 [#1] PREEMPT SMP KASAN
> CPU: 0 PID: 5068 Comm: syz-executor357 Not tainted 6.6.0-syzkaller-16039-gac347a0655db #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/09/2023
> RIP: 0010:drm_mode_vrefresh drivers/gpu/drm/drm_modes.c:1303 [inline]
> RIP: 0010:drm_mode_debug_printmodeline+0x118/0x4e0 drivers/gpu/drm/drm_modes.c:60
> Code: 00 41 0f b7 07 66 83 f8 02 b9 01 00 00 00 0f 43 c8 0f b7 c1 0f af e8 44 89 f0 48 69 c8 e8 03 00 00 89 e8 d1 e8 48 01 c8 31 d2 <48> f7 f5 49 89 c6 eb 0c e8 fb 07 66 fc eb 05 e8 f4 07 66 fc 48 89
> RSP: 0018:ffffc9000391f8d0 EFLAGS: 00010246
> RAX: 000000000001f400 RBX: ffff888025045000 RCX: 000000000001f400
> RDX: 0000000000000000 RSI: 0000000000008000 RDI: ffff888025045018
> RBP: 0000000000000000 R08: ffffffff8528b9af R09: 0000000000000000
> R10: ffffc9000391f8a0 R11: fffff52000723f17 R12: 0000000000000080
> R13: dffffc0000000000 R14: 0000000000000080 R15: ffff888025045016
> FS:  0000555556932380(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00000000005fdeb8 CR3: 000000007fcff000 CR4: 00000000003506f0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  <TASK>
>  drm_mode_setcrtc+0x83b/0x1880 drivers/gpu/drm/drm_crtc.c:794
>  drm_ioctl_kernel+0x362/0x500 drivers/gpu/drm/drm_ioctl.c:792
>  drm_ioctl+0x636/0xb00 drivers/gpu/drm/drm_ioctl.c:895
>  vfs_ioctl fs/ioctl.c:51 [inline]
>  __do_sys_ioctl fs/ioctl.c:871 [inline]
>  __se_sys_ioctl+0xf8/0x170 fs/ioctl.c:857
>  do_syscall_x64 arch/x86/entry/common.c:51 [inline]
>  do_syscall_64+0x44/0x110 arch/x86/entry/common.c:82
>  entry_SYSCALL_64_after_hwframe+0x63/0x6b
> 
> [Analysis]
> When calculating den in drm_mode_vrefresh(), if the vscan value is too large, 
> there is a probability of unsigned integer overflow.
> 
> [Fix]
> Before multiplying by vscan, first check if their product will overflow. 
> If overflow occurs, return 0 and exit the subsequent process.
> 
> Reported-and-tested-by: syzbot+2e93e6fb36e6fdc56574 at syzkaller.appspotmail.com
> Fixes: ea40d7857d52 ("drm/vkms: fbdev emulation support")
> Signed-off-by: Edward Adam Davis <eadavis at qq.com>
> ---
>  drivers/gpu/drm/drm_modes.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletion(-)
> 
> diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
> index ac9a406250c5..60739d861da2 100644
> --- a/drivers/gpu/drm/drm_modes.c
> +++ b/drivers/gpu/drm/drm_modes.c
> @@ -36,6 +36,7 @@
>  #include <linux/list.h>
>  #include <linux/list_sort.h>
>  #include <linux/of.h>
> +#include <linux/overflow.h>
>  
>  #include <video/of_display_timing.h>
>  #include <video/of_videomode.h>
> @@ -1297,8 +1298,10 @@ int drm_mode_vrefresh(const struct drm_display_mode *mode)
>  		num *= 2;
>  	if (mode->flags & DRM_MODE_FLAG_DBLSCAN)
>  		den *= 2;
> -	if (mode->vscan > 1)
> -		den *= mode->vscan;
> +	if (mode->vscan > 1) {
> +		if (unlikely(check_mul_overflow(den, mode->vscan, &den)))
> +			return 0;
> +	}

I can't see any driver that actually supports vscan>1. Only
nouveau has some code for it, but doesn't look like it does
anything sensible. All other drivers for sure should be
rejecting vscan>1 outright. Which driver is this?

Is there an actual usecase where nouveau needs this (and does
it even work?) or could we just rip out the whole thing and
reject vscan>1 globally?

>  
>  	return DIV_ROUND_CLOSEST_ULL(mul_u32_u32(num, 1000), den);
>  }
> -- 
> 2.25.1

-- 
Ville Syrjälä
Intel