[Mesa-dev] [PATCH] st/wgl: Clamp wglChoosePixelFormatARB's output nNumFormats to nMaxFormats.

[Brian Paul]

On 07/24/2014 08:51 AM, jfonseca at vmware.com wrote:
> From: José Fonseca <jfonseca at vmware.com>
>
> While running https://github.com/nvMcJohn/apitest with apitrace I noticed that Mesa was producing bogus results:
>
>    wglChoosePixelFormatARB(hdc, piAttribIList = {...}, pfAttribFList = &0, nMaxFormats = 1, piFormats = {19, 65576, 37, 198656, 131075, 0, 402653184, 0, 0, 0, 0, -573575710}, nNumFormats = &12) = TRUE
>
> However https://www.opengl.org/registry/specs/ARB/wgl_pixel_format.txt states
>
>      <nNumFormats> returns the number of matching formats. The returned
>      value is guaranteed to be no larger than <nMaxFormats>.
>
> Cc: "10.2" <mesa-stable at lists.freedesktop.org>
> ---
>   src/gallium/state_trackers/wgl/stw_ext_pixelformat.c | 6 ++++--
>   1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/src/gallium/state_trackers/wgl/stw_ext_pixelformat.c b/src/gallium/state_trackers/wgl/stw_ext_pixelformat.c
> index d303b01..91682d1 100644
> --- a/src/gallium/state_trackers/wgl/stw_ext_pixelformat.c
> +++ b/src/gallium/state_trackers/wgl/stw_ext_pixelformat.c
> @@ -448,9 +448,11 @@ wglChoosePixelFormatARB(
>       */
>      for (i = 0; i < count; i++) {
>         if (scores[i].points > 0) {
> -         if (*nNumFormats < nMaxFormats)
> -            piFormats[*nNumFormats] = scores[i].index + 1;
> +	 piFormats[*nNumFormats] = scores[i].index + 1;
>            (*nNumFormats)++;
> +	 if (*nNumFormats >= nMaxFormats) {
> +	    break;
> +	 }
>         }
>      }
>
>


Reviewed-by: Brian Paul <brianp at vmware.com>