[Intel-gfx] [Draft] Testing Requirements for drm/i915 Patches

[Jesse Barnes]

On Tue, 29 Oct 2013 20:00:49 +0100
Daniel Vetter <daniel at ffwll.ch> wrote:

> Since the point here is to make the actual test requirements known up-front we
> need to settle on clear expectations. Since this is the part that actually
> matters in practice I'll really welcome close scrutiny and comments here.

Another thought on these lines.

The expectations come from more than just the test side of things, but
also from the design of new features, or how code is refactored for a
new platform or to fix a bug.

So it may make sense, before starting big work, to propose both the
tests required for the feature/refactor/bug fix, as well as the
proposed design of the change.  That would let us get any potential
test & feature bikeshedding out of the way before tons of time is
invested in the code, hopefully saving a lot of rage, especially on the
big features.

Note that this runs contrary to a lot of other subsystems, which are
sometimes allergic to up front design thinking, and prefer to churn
through dozens of implementations to settle on something.  Both
approaches have value, and some combination is probably best.  Some
well thought out discussion up front, followed by testing, review, and
adjustment of an actual implementation.

Getting back to your points:
  - tests must cover userspace interfaces
  - tests need to provide coverage of internal driver state
  - tests need to be written following review for any bugs caught in
    review (I don't fully agree here; we can look at this on a case by
    case basis; e.g. in some cases an additional BUG_ON or state check
    may be all that's needed)
  - test infrastructure and scope should increase over time

I think we can discuss the above points as part of any new proposed
feature or rework.  For reworks in particular, we can probably start to
address some of the "technical debt" you mentioned, where the bar for a
rework is relatively high, requiring additional tests and
infrastructure scope.

-- 
Jesse Barnes, Intel Open Source Technology Center