[PATCH] drm/radeon/r100: Fix integer overflow issues in r100_cs_track_check()

[Alex Deucher]

Applied.  Thanks!

On Wed, Nov 29, 2023 at 10:28 AM Nikita Zhandarovich
<n.zhandarovich at fintech.ru> wrote:
>
> It may be possible, albeit unlikely, to encounter integer overflow
> during the multiplication of several unsigned int variables, the
> result being assigned to a variable 'size' of wider type.
>
> Prevent this potential behaviour by converting one of the multiples
> to unsigned long.
>
> Found by Linux Verification Center (linuxtesting.org) with static
> analysis tool SVACE.
>
> Fixes: 0242f74d29df ("drm/radeon: clean up CS functions in r100.c")
> Signed-off-by: Nikita Zhandarovich <n.zhandarovich at fintech.ru>
> ---
>  drivers/gpu/drm/radeon/r100.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/radeon/r100.c b/drivers/gpu/drm/radeon/r100.c
> index affa9e0309b2..cfeca2694d5f 100644
> --- a/drivers/gpu/drm/radeon/r100.c
> +++ b/drivers/gpu/drm/radeon/r100.c
> @@ -2321,7 +2321,7 @@ int r100_cs_track_check(struct radeon_device *rdev, struct r100_cs_track *track)
>         switch (prim_walk) {
>         case 1:
>                 for (i = 0; i < track->num_arrays; i++) {
> -                       size = track->arrays[i].esize * track->max_indx * 4;
> +                       size = track->arrays[i].esize * track->max_indx * 4UL;
>                         if (track->arrays[i].robj == NULL) {
>                                 DRM_ERROR("(PW %u) Vertex array %u no buffer "
>                                           "bound\n", prim_walk, i);
> @@ -2340,7 +2340,7 @@ int r100_cs_track_check(struct radeon_device *rdev, struct r100_cs_track *track)
>                 break;
>         case 2:
>                 for (i = 0; i < track->num_arrays; i++) {
> -                       size = track->arrays[i].esize * (nverts - 1) * 4;
> +                       size = track->arrays[i].esize * (nverts - 1) * 4UL;
>                         if (track->arrays[i].robj == NULL) {
>                                 DRM_ERROR("(PW %u) Vertex array %u no buffer "
>                                           "bound\n", prim_walk, i);
> --
> 2.25.1
>