[Mesa-dev] [RFC] Merging feature work to Mesa master

[Ian Romanick]

Over the last six months a lot of feature work has happened in Mesa, and 
the load has been carried by a lot of different people / organization. 
In the process, we discovered a number of development process issues 
that made things more difficult than they needed to be.

The biggest problem the we encountered was the number of features marked 
"50% done" in GL3.txt.  To complete these features, a significant amount 
of time had to be spent figuring out what was done, and what was left to 
do.  Since the changes had landed on master a very long time ago, this 
was a frustrating and error prone process.  Transform feedback was the 
worst case of this.  One developer spent several weeks trying to assess 
the state of development.  In the process, a couple items were missed 
and not detected until early January.

PROPOSAL: No feature will be merged to master until a vertical slice[1] 
of functionality is complete.

To be specific, this means that some useful workload should be able to 
correctly execute.  This doesn't mean that everything is done or even 
that any particular real application needs to work.  At the very least 
there should be a demo or piglit test that uses the functionality in its 
intended manner that works.

This means that some incomplete features may sit on branches of a long 
time.  That's okay!  It's really easy to see what has been done on a 
branch because you can 'git log origin/master..EXT_some_feature'.  This 
trivializes the assessment of the state of development.

We encountered similar problems with pieces of functionality that were 
ostensibly done.  Many elements of OpenGL functionality are like Koch 
snowflakes:  everything is a corner case.  Integer textures and 
floating-point textures are prime cases of this.  Even though the 
implementation was done and enabled in several drivers, we had no way to 
assess the quality.  The same problem holds in cases of features that 
are known to be incomplete, even if a vertical slice is functional.

PROPOSAL: No feature will be merged to master until a robust set of 
tests are implemented or at least documented.

We don't necessarily need 10,000 tests for some feature, but there 
should be some.  There should also be a list of "test this corner case, 
test that corner case, check this error condition, etc."  As an example, 
we've come up with a list of missing test cases for 
EXT_framebuffer_multisample:

- Test turning multisample on and off on a MSAA buffer.
- Test multisample points smooth
- Test multisample points non-smooth
- Test multisample lines smooth
- Test multisample lines non-smooth
- Test multisample line stipple
- Test multisample polygon smooth
- Test multisample polygon stipple
- Test multisample glBitmap
- Test sample alpha to one
- Test sample coverage
- Test sample coverage invert
- Test sample alpha-to-coverage, with alpha-to-one
- Test sample alpha-to-coverage, without alpha-to-one
- Test blit multisample-to-multisample (matching sample count)
- Test blit multisample-to-multisample (non-matching sample count)
- Test blit multisample-to-multisample (non-matching format)
- Test blit singlesample-to-multisample
- Test blit multisample-to-multisample (scaling blit)
- Iterate all the above for all the sample counts
- Iterate all internal formats for some basic MSAA test
- "All of the above" for depth and stencil
     - Try really hard to hit HiZ resolves too

This list represents about 30 or 40 test cases.  By giving some 
indication of areas that need testing, we communicate the parts of the 
feature that are known to be incomplete or just plain broken (Keith's 
1st Law: Any piece of code that isn't tested is broken.).

The test list also need not necessarily come from the developer.  It is 
completely reasonable and desirable for reviewers to suggest or 
implement tests.  All of that information just needs to be captured, and 
consensus must be achieved.

The onus of enforcing these proposals will fall on the reviewers.  This 
still leaves room for sensible things to happen while preventing the 
problems that we encountered.  For example, it should be fine to merge 
Marek and nobled's ARB_debug work with few tests implemented or 
specified.  The same could not be said for the proposed implementation 
of ARB_uniform_buffer_object.

[1] http://en.wikipedia.org/wiki/Vertical_slice