[Mesa-dev] stack overflow in gluEndSurface

[Brian Paul]

On 12/08/2010 07:10 PM, Raymond Maple wrote:
> Hello,
>      I have run into a problem with gluNurbs that I would appreciate
> some help with.  I am attempting to render trimmed NURBS surfaces read
> from an IGES file.  The first few surfaces seem to render OK, but after
> some number, say N, I start getting a GL_STACK_OVERFLOW error with the
> call to gluEndSurface.  If I render any single trimmed surface, it works
> fine, so I don't think its the NURBS data.  Also, if I render just the
> untrimmed surface, everything is fine, which points to something in the
> trimming code.   I've checked the depth of the modelview, projection,
> and texture matrix stacks before and after the glEndSurface call, and
> they are all what I expect them to be (1 or 2), so the error must be
> thrown by something in the glu nurbs tesselator.  I've searched through
> the glu code, but haven't found anywhere that the matrix stacks are
> manipulated, or where GL_STACK_OVERFLOW is thrown, though I did see
> where it is included in libutil/error.cc  If anyone could shed some
> light on what might be going on, I'd appreciate it.
> Additional data:
> Problem seen with GLU libraries shipped with Ubuntu 9.04 and RHEL 5.4
> (not sure of versions)
> Nvidia drivers & core OpenGL
> pyOpenGL 3.0/3.1
> All surfaces and trim curves are non-rational BSplines
> I'm rendering using object space sampling, but have same error with
> default pixel sampling
> Onset of error seems to depend on sampling size, but once one surface
> has a problem, all following surfaces do.

The GLU code calls glPush/PopAttrib() in a few places.  You might look 
for those and see if there's a potential mismatch somewhere.

Otherwise, if you could use a debug build of Mesa you could put a 
breakpoint on _mesa_error() to see where the error is getting generated.

-Brain