[Mesa-dev] [PATCH 4/8] draw/gs: fix segfault in glsl-1.50-gs-mismatch-prim-type triangles_adjacency

[Dave Airlie]

On 11 June 2014 06:23, Zack Rusin <zackr at vmware.com> wrote:
> I think the code is already correct and something else goes wrong. The tgsi geometry shader code was never done properly so it's more than likely that tgsi_exec is doing something wonky.
>
> Geometry shaders specify the maximum number of vertices that they can emit. That's what draw_geometry_shader::max_output_vertices is. If a geometry shader emits more than that, the verts will be ignored. So our primitive_boundary is max_output_vertices + 1  because we want to make sure that in SoA we have a scratch space where we can keep writing the overflowed vertices.
>
> So the worst case scenario for our output buffer is: (max_output_vertices + 1) * geometry shader invocations. That's what we have there now and that's correct. I don't remember what tgsi_exec does, I think I never even implemented proper SoA for gs in tgsi_exec, so if there's anything wrong I'd look for the bug there.
>

Well the main difference is the vector_length is 1 for TGSI, and 4 for
LLVM, this seems to cover over the problem I've seen here,

I'll revisit it today and see if I can spot something else wrong, it
fails for triangle adj because there are 6 vertices per primitive and
we have only malloced space for 4.

Dave.