Mesa (master): st/mesa: fix accum buffer allocation in st_renderbuffer_alloc_storage()

[Brian Paul]

Module: Mesa
Branch: master
Commit: c87d1a3c3d24924f8af082e47c18a6d535419089
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=c87d1a3c3d24924f8af082e47c18a6d535419089

Author: Brian Paul <brianp at vmware.com>
Date:   Mon Nov 21 13:59:35 2011 -0700

st/mesa: fix accum buffer allocation in st_renderbuffer_alloc_storage()

If the gallium driver doesn't support PIPE_FORMAT_R16G16B16A16_SNORM
the call to st_choose_renderbuffer_format() would fail and we'd generate
an GL_OUT_OF_MEMORY error.  We'd never get to the subsequent code that
handles software/malloc-based renderbuffers.

Add a special-case check for PIPE_FORMAT_R16G16B16A16_SNORM which is used
for software-based accum buffers.  This could be fixed in other ways but
it would be a much larger patch.  st_renderbuffer_alloc_storage() could
be reorganized in the future.

This fixes accum buffer allocation for the svga driver.

Note: This is a candidate for the 7.11 branch.

Reviewed-by: José Fonseca <jfonseca at vmware.com>

---

 src/mesa/state_tracker/st_cb_fbo.c |   14 ++++++++++++--
 1 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/src/mesa/state_tracker/st_cb_fbo.c b/src/mesa/state_tracker/st_cb_fbo.c
index a351d11..1e8ec05 100644
--- a/src/mesa/state_tracker/st_cb_fbo.c
+++ b/src/mesa/state_tracker/st_cb_fbo.c
@@ -75,8 +75,18 @@ st_renderbuffer_alloc_storage(struct gl_context * ctx,
    enum pipe_format format;
    struct pipe_surface surf_tmpl;
 
-   format = st_choose_renderbuffer_format(screen, internalFormat,
-                                          rb->NumSamples);
+   if (internalFormat == GL_RGBA16_SNORM && strb->software) {
+      /* Special case for software accum buffers.  Otherwise, if the
+       * call to st_choose_renderbuffer_format() fails (because the
+       * driver doesn't support signed 16-bit/channel colors) we'd
+       * just return without allocating the software accum buffer.
+       */
+      format = PIPE_FORMAT_R16G16B16A16_SNORM;
+   }
+   else {
+      format = st_choose_renderbuffer_format(screen, internalFormat,
+                                             rb->NumSamples);
+   }
 
    if (format == PIPE_FORMAT_NONE) {
       return FALSE;