[Mesa-dev] [PATCH] mesa: sanitize input stencil mask values

[Haixia Shi]

Hello Mesa-Dev:

To give you some background, we are observing that mask values getting
converted to -1 when running multiple dEQP-GLES31 tests in sequence.

There's already a previous commit
(b8b1d83c71fd148d2fd84afdc20c0aa367114f92) that attempted to address the
test failures by initializing all stencil values to 0xFF.

However, the test still fails when running all
dEQP-GLES3.functional.state_query.integers#stencil_*_getfloat tests
sequentially.

Apparently the test sets mask values to 0xffffffffU using glStencilFunc()
API, and this overwrites the 0xFF initial value. Therefore I would propose
that we sanitize all input stencil mask values.

Thanks,
Haixia

On Thu, Dec 22, 2016 at 10:43 AM, Haixia Shi <hshi at chromium.org> wrote:

> Since the maximum supported precision for stencil buffers is 8 bits, we
> should set bits 8:31 in the input mask values to zero.
>
> The problem is that dEQP tests intentionally call glStencilFunc() with mask
> values of ~0U (0xffffffffU) which overflows to -1 when converted to signed
> integer by glGet* APIs.
>
> Fixes 6 dEQP failing tests:
> * dEQP-GLES3.functional.state_query.integers.stencil_value_mask_getfloat
> * dEQP-GLES3.functional.state_query.integers.stencil_back_valu
> e_mask_getfloat
> * dEQP-GLES3.functional.state_query.integers.stencil_value_mas
> k_separate_getfloat
> * dEQP-GLES3.functional.state_query.integers.stencil_value_mas
> k_separate_both_getfloat
> * dEQP-GLES3.functional.state_query.integers.stencil_back_valu
> e_mask_separate_getfloat
> * dEQP-GLES3.functional.state_query.integers.stencil_back_valu
> e_mask_separate_both_getfloat
>
> Signed-off-by: Haixia Shi <hshi at chromium.org>
> Cc: Ian Romanick <ian.d.romanick at intel.com>
> ---
>  src/mesa/main/stencil.c | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
>
> diff --git a/src/mesa/main/stencil.c b/src/mesa/main/stencil.c
> index b303bb7..f29187c 100644
> --- a/src/mesa/main/stencil.c
> +++ b/src/mesa/main/stencil.c
> @@ -198,6 +198,11 @@ _mesa_StencilFunc( GLenum func, GLint ref, GLuint
> mask )
>        return;
>     }
>
> +   /* Since the maximum supported precision for stencil buffers is 8 bits,
> +    * set bits 8:31 in the input mask value to zero.
> +    */
> +   mask &= 0xFF;
> +
>     if (face != 0) {
>        if (ctx->Stencil.Function[face] == func &&
>            ctx->Stencil.ValueMask[face] == mask &&
> @@ -258,6 +263,11 @@ _mesa_StencilMask( GLuint mask )
>     if (MESA_VERBOSE & VERBOSE_API)
>        _mesa_debug(ctx, "glStencilMask()\n");
>
> +   /* Since the maximum supported precision for stencil buffers is 8 bits,
> +    * set bits 8:31 in the input mask value to zero.
> +    */
> +   mask &= 0xFF;
> +
>     if (face != 0) {
>        /* Only modify the EXT_stencil_two_side back-face state.
>         */
> @@ -468,6 +478,11 @@ _mesa_StencilFuncSeparate(GLenum face, GLenum func,
> GLint ref, GLuint mask)
>
>     FLUSH_VERTICES(ctx, _NEW_STENCIL);
>
> +   /* Since the maximum supported precision for stencil buffers is 8 bits,
> +    * set bits 8:31 in the input mask value to zero.
> +    */
> +   mask &= 0xFF;
> +
>     if (face != GL_BACK) {
>        /* set front */
>        ctx->Stencil.Function[0] = func;
> @@ -502,6 +517,11 @@ _mesa_StencilMaskSeparate(GLenum face, GLuint mask)
>
>     FLUSH_VERTICES(ctx, _NEW_STENCIL);
>
> +   /* Since the maximum supported precision for stencil buffers is 8 bits,
> +    * set bits 8:31 in the input mask value to zero.
> +    */
> +   mask &= 0xFF;
> +
>     if (face != GL_BACK) {
>        ctx->Stencil.WriteMask[0] = mask;
>     }
> --
> 2.8.0.rc3.226.g39d4020
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20161222/b8991441/attachment.html>