[Mesa-dev] [PATCH v2] mesa: Identify packed depth/stencil buffers using the Format field.

[Nick Bowler]

Intel sometimes uses packed depth/stencil buffers even when only a depth
buffer or only a stencil buffer was requested.  Common code currently
uses the _BaseFormat field to determine whether a depth/stencil wrapper
is necessary.  But unless the user explicitly requested a packed
depth/stencil buffer, the _BaseFormat field does not encode this
information, and the required wrappers are not created.

The problem was introduced by commit 45e76d2665b38b ("mesa: remove a
bunch of gl_renderbuffer fields"), which killed off the _ActualFormat
field upon which the decision to create a wrapper used to be made.  This
patch changes the logic to use the Format field instead, which is more
like the old code.

Fixes fdo bug 27590.

Signed-off-by: Nick Bowler <nbowler at draconx.ca>
---
 v2: Rewrite the helper function as a generic format helper.
     Suggested by Brian Paul.

 src/mesa/main/formats.c     |   15 +++++++++++++++
 src/mesa/main/formats.h     |    3 +++
 src/mesa/main/framebuffer.c |    4 ++--
 3 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/src/mesa/main/formats.c b/src/mesa/main/formats.c
index 90449cc..464b93f 100644
--- a/src/mesa/main/formats.c
+++ b/src/mesa/main/formats.c
@@ -940,6 +940,21 @@ _mesa_is_format_compressed(gl_format format)
 
 
 /**
+ * Determine if the given format represents a packed depth/stencil buffer.
+ */
+GLboolean
+_mesa_is_format_packed_depth_stencil(gl_format format)
+{
+   if (format == MESA_FORMAT_Z24_S8
+       || format == MESA_FORMAT_Z24_X8
+       || format == MESA_FORMAT_S8_Z24
+       || format == MESA_FORMAT_X8_Z24)
+      return 1;
+
+   return 0;
+}
+
+/**
  * Return color encoding for given format.
  * \return GL_LINEAR or GL_SRGB
  */
diff --git a/src/mesa/main/formats.h b/src/mesa/main/formats.h
index ad176ca..faa922f 100644
--- a/src/mesa/main/formats.h
+++ b/src/mesa/main/formats.h
@@ -190,6 +190,9 @@ _mesa_get_format_block_size(gl_format format, GLuint *bw, GLuint *bh);
 extern GLboolean
 _mesa_is_format_compressed(gl_format format);
 
+extern GLboolean
+_mesa_is_format_packed_depth_stencil(gl_format format)
+
 extern GLenum
 _mesa_get_format_color_encoding(gl_format format);
 
diff --git a/src/mesa/main/framebuffer.c b/src/mesa/main/framebuffer.c
index e0aac26..3099fc3 100644
--- a/src/mesa/main/framebuffer.c
+++ b/src/mesa/main/framebuffer.c
@@ -611,7 +611,7 @@ _mesa_update_depth_buffer(GLcontext *ctx,
 
    depthRb = fb->Attachment[attIndex].Renderbuffer;
 
-   if (depthRb && depthRb->_BaseFormat == GL_DEPTH_STENCIL) {
+   if (depthRb && _mesa_format_is_packed_depth_stencil(depthRb->Format)) {
       /* The attached depth buffer is a GL_DEPTH_STENCIL renderbuffer */
       if (!fb->_DepthBuffer
           || fb->_DepthBuffer->Wrapped != depthRb
@@ -652,7 +652,7 @@ _mesa_update_stencil_buffer(GLcontext *ctx,
 
    stencilRb = fb->Attachment[attIndex].Renderbuffer;
 
-   if (stencilRb && stencilRb->_BaseFormat == GL_DEPTH_STENCIL) {
+   if (stencilRb && _mesa_format_is_packed_depth_stencil(stencilRb->Format)) {
       /* The attached stencil buffer is a GL_DEPTH_STENCIL renderbuffer */
       if (!fb->_StencilBuffer
           || fb->_StencilBuffer->Wrapped != stencilRb
-- 
1.7.1