[PATCH] gpu/drm: fix signed integer overflow

[Sean Paul]

On Tue, Sep 6, 2016 at 4:55 AM, Xie XiuQi <xiexiuqi at huawei.com> wrote:
> Use 1UL for unsigned long, or we'll meet a overflow issue with UBSAN.
>
> [   15.589489] UBSAN: Undefined behaviour in drivers/gpu/drm/drm_hashtab.c:145:35
> [   15.589500] signed integer overflow:
> [   15.589999] -2147483648 - 1 cannot be represented in type 'int'
> [   15.590434] CPU: 2 PID: 294 Comm: plymouthd Not tainted 3.10.0-327.28.3.el7.x86_64 #1
> [   15.590653] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/07/2011
> [   15.591001]  1ffff1000670fe83 000000000d6b385e ffff88003387f3e0 ffffffff81ee3140
> [   15.591028]  ffff88003387f3f8 ffffffff81ee31fd ffffffffa032f460 ffff88003387f560
> [   15.591044]  ffffffff81ee46e2 0000002d00000009 0000000000000001 0000000041b58ab3
> [   15.591059] Call Trace:
> [   15.591078]  [<ffffffff81ee3140>] dump_stack+0x1e/0x20
> [   15.591093]  [<ffffffff81ee31fd>] ubsan_epilogue+0x12/0x55
> [   15.591109]  [<ffffffff81ee46e2>] handle_overflow+0x1ba/0x215
> [   15.591126]  [<ffffffff81ee4528>] ? __ubsan_handle_negate_overflow+0x162/0x162
> [   15.591146]  [<ffffffff8103416c>] ? print_context_stack+0x9c/0x160
> [   15.591163]  [<ffffffff81031df2>] ? dump_trace+0x252/0x750
> [   15.591181]  [<ffffffff81739023>] ? __list_add+0x93/0x160
> [   15.591197]  [<ffffffff81ee4798>] __ubsan_handle_sub_overflow+0x2a/0x31
> [   15.591261]  [<ffffffffa0282140>] drm_ht_just_insert_please+0x1e0/0x200 [drm]
> [   15.591290]  [<ffffffffa0528c7a>] ttm_base_object_init+0x10a/0x270 [ttm]
> [   15.591316]  [<ffffffffa052a34c>] ttm_vt_lock+0x28c/0x3a0 [ttm]
> [   15.591343]  [<ffffffffa052a0c0>] ? ttm_write_lock+0x180/0x180 [ttm]
> [   15.591362]  [<ffffffff81419526>] ? kasan_unpoison_shadow+0x36/0x50
> [   15.591379]  [<ffffffff81419526>] ? kasan_unpoison_shadow+0x36/0x50
> [   15.591396]  [<ffffffff81419526>] ? kasan_unpoison_shadow+0x36/0x50
> [   15.591413]  [<ffffffff81419526>] ? kasan_unpoison_shadow+0x36/0x50
> [   15.591442]  [<ffffffffa061cbe1>] vmw_master_set+0x121/0x470 [vmwgfx]
> [   15.591459]  [<ffffffff811773a5>] ? __init_waitqueue_head+0x45/0x70
> [   15.591487]  [<ffffffffa061cac0>] ? vmw_master_drop+0x310/0x310 [vmwgfx]
> [   15.591535]  [<ffffffffa026946a>] drm_open+0x92a/0xc00 [drm]
> [   15.591563]  [<ffffffffa0619ff0>] ? vmw_driver_open+0x170/0x170 [vmwgfx]
> [   15.591610]  [<ffffffffa0268b40>] ? drm_poll+0xe0/0xe0 [drm]
> [   15.591661]  [<ffffffffa02797b4>] drm_stub_open+0x224/0x330 [drm]
> [   15.591711]  [<ffffffffa0279590>] ? drm_minor_acquire+0x240/0x240 [drm]
> [   15.591727]  [<ffffffff8145fa8a>] chrdev_open+0x1fa/0x3f0
> [   15.591742]  [<ffffffff8145f890>] ? cdev_put+0x50/0x50
> [   15.591761]  [<ffffffff814f6dc3>] ? __fsnotify_parent+0x53/0x210
> [   15.591778]  [<ffffffff8144fde1>] do_dentry_open+0x351/0x670
> [   15.591792]  [<ffffffff8145f890>] ? cdev_put+0x50/0x50
> [   15.591807]  [<ffffffff814503c2>] vfs_open+0xa2/0x170
> [   15.591824]  [<ffffffff8147b5df>] do_last+0xccf/0x2c80
> [   15.591842]  [<ffffffff8147a910>] ? filename_create+0x320/0x320
> [   15.591858]  [<ffffffff81472549>] ? path_init+0x1b9/0xa90
> [   15.591875]  [<ffffffff81472390>] ? mountpoint_last+0x9a0/0x9a0
> [   15.591894]  [<ffffffff815f9ccf>] ? selinux_file_alloc_security+0xcf/0x130
> [   15.591911]  [<ffffffff8147d777>] path_openat+0x1e7/0xcc0
> [   15.591927]  [<ffffffff81031df2>] ? dump_trace+0x252/0x750
> [   15.591943]  [<ffffffff8147d590>] ? do_last+0x2c80/0x2c80
> [   15.591959]  [<ffffffff81739023>] ? __list_add+0x93/0x160
> [   15.591974]  [<ffffffff8104b48d>] ? save_stack_trace+0x7d/0xb0
> [   15.591989]  [<ffffffff81480824>] do_filp_open+0xa4/0x160
> [   15.592004]  [<ffffffff81480780>] ? user_path_mountpoint_at+0x50/0x50
> [   15.592022]  [<ffffffff8149d755>] ? __alloc_fd+0x175/0x300
> [   15.592039]  [<ffffffff81453127>] do_sys_open+0x1b7/0x3f0
> [   15.592054]  [<ffffffff81452f70>] ? filp_open+0x80/0x80
> [   15.592070]  [<ffffffff81453392>] SyS_open+0x32/0x40
> [   15.592088]  [<ffffffff81f08989>] system_call_fastpath+0x16/0x1b
>
> Signed-off-by: Xie XiuQi <xiexiuqi at huawei.com>

Applied to drm-misc (with slightly tweaked subject)

> ---
>  drivers/gpu/drm/drm_hashtab.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/drm_hashtab.c b/drivers/gpu/drm/drm_hashtab.c
> index 7b30b30..dae18e5 100644
> --- a/drivers/gpu/drm/drm_hashtab.c
> +++ b/drivers/gpu/drm/drm_hashtab.c
> @@ -142,7 +142,7 @@ int drm_ht_just_insert_please(struct drm_open_hash *ht, struct drm_hash_item *it
>                               unsigned long add)
>  {
>         int ret;
> -       unsigned long mask = (1 << bits) - 1;
> +       unsigned long mask = (1UL << bits) - 1;
>         unsigned long first, unshifted_key;
>
>         unshifted_key = hash_long(seed, bits);
> --
> 1.8.3.1
>