[Mesa-dev] first attempt at shader stencil export

[Brian Paul]

On 09/30/2010 12:41 AM, Dave Airlie wrote:
> some background:
>
> so on r600g, the only way to directly write to the stencil is via the
> shader, using a transfer would require an extra step to flush the DS
> buffer out via the pipe again to make it useable by the hw as a DS
> buffer. So using the current gallium stencil draw would be a major
> overhead, compared to just doing it properly.
>
> So to do it properly I decided to expose the
> GL_ARB_shader_stencil_export type functionality.
>
> http://cgit.freedesktop.org/~airlied/mesa/log/?h=gallium-stencil-export
>
> 7 commits in here:
>
> two simple one liners, add the cap to gallium headers, and add
> FRAG_RESULT_STENCIL to mesa.
> then a "fix" to the mesa texstore so it will store S8 in an 8/24 so we
> can put the stencil values in a texture.
> mesa state tracker support to use the new capability to hw accel
> drawpixels on the stencil (outputs to Y of FRAG_RESULT_STENCIL).
> r600g support for the capability take the second
> TGSI_SEMANTIC_POSITION output and use its Y value as stencil (looking
> at this now, I should probably be taking the X value really).
> very initial mesa/GLSL support for the extension (completely untested).
> initial softpipe support for the capability a demonstration.
>
> TODO:
> probably stop using Y (this was just because the hw uses Y), the GLSL
> extension just defines an integer output for the fragprog.
> fix the 24/8 texstore variant.
> write some test code for piglit and test the GL extension/GLSL bits.
>
> I'm a lot more interested in the non-GL extension bits as it allows
> stencil writes to work properly on r600g.

A few comments.

Instead of using semantic name=TGSI_SEMANTIC_POSITION, index=1 to 
indicate the stencil output, how about just defining a new 
TGSI_SEMANTIC_STENCIL label instead?  I think that'd be clearer.

I think the fragment stencil result is supposed to be an integer.  But 
in the softpipe changes, it looks like you're doing a [0,1]->[0,255] 
scaling at one point.

If we want to use this feature for implementing glDraw/CopyPixels 
functionality, the stencil values will be coming from a texture.  How 
do we sample that texture?  That is, which components of the texture 
image return the Z and/or stencil components?  Basically, we need to 
add a new row to the table in the tgsi.rst file for Z/Stencil 
textures.  Also, we need to clarify that fetching a stencil value 
returns an integer value, not a float normalized to [0,1].

In softpipe, there's some logic to determine whether we can do depth 
testing before fragment shading (we can't when the shader writes Z). 
That needs to be updated to check if the shader writes stencil.

Thanks for doing this, Dave!

-Brian