[Mesa-dev] [Bug 102038] assertion failure in update_framebuffer_size

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=102038

--- Comment #8 from Bruce Cherniak <bruce.cherniak at intel.com> ---
Roland, you are correct, VTK is trying to use msaa surfaces with just one
sample.

Here's what's happening:

The rendertarget color buffer is being created by:
st_api_make_current
  st_framebuffer_validate
    xmesa_st_frambuffer_validate_textures
      llvmpipe_resource_create
with nr_samples = 1

Then later in st_framebuffer_validate "changed" is set when the surface is
created.  This causes the following:

_mesa_resize_framebuffer  (loops until it gets to the depth attachment)
  st_renderbuffer_alloc_storage

until we get to:
160├──> if (rb->NumSamples > 0) {
161│       unsigned i;
162│
163│       for (i = MAX2(2, rb->NumSamples); i <= ctx->Const.MaxSamples; i++) {
164│          format = st_choose_renderbuffer_format(st, internalFormat, i);
165│
166│          if (format != PIPE_FORMAT_NONE) {
167│             rb->NumSamples = i;
168│             break;
169│          }
170│       }
171│    } else {
172│       format = st_choose_renderbuffer_format(st, internalFormat, 0);
173│    }
174│
175│    /* Not setting gl_renderbuffer::Format here will cause
176│     * FRAMEBUFFER_UNSUPPORTED and ValidateFramebuffer will not be called.
177│     */
178|    if (format == PIPE_FORMAT_NONE) {
179│       return GL_TRUE;
180│    }

rb->NumSamples = 1, but ctx->Const.MaxSamples = 0 so the loop @ 163 is skipped
and format remains  PIPE_FORMAT_NONE.  Because to this, we don't allocate a
depth buffer and boom.

But, this works:
-   if (rb->NumSamples > 0) {
+   if (rb->NumSamples > 1) {
       unsigned i;

       for (i = MAX2(2, rb->NumSamples); i <= ctx->Const.MaxSamples; i++) {
@@ -169,7 +170,7 @@ st_renderbuffer_alloc_storage(struct gl_context * ctx,
          }
       }
    } else {
-      format = st_choose_renderbuffer_format(st, internalFormat, 0);
+      format = st_choose_renderbuffer_format(st, internalFormat,
rb->NumSamples);
    }

I'm not sure if it honors the original intent.  But, in addition to fixing the
VTK tests ext_framebuffer_multisample-blit-mismatched-formats now passes.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are the QA Contact for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20170821/84adc1f4/attachment.html>