A modest start at a test suite for apitrace

[José Fonseca]

Hi Carl,

On Mon, Apr 9, 2012 at 11:43 PM, Carl Worth <cworth at cworth.org> wrote:
> I'm (finally) picking up my old "apitrace trim" work to make it actually
> interesting this time, (with some solid dependency analysis, etc.).
>
> Before I can trust some of the changes I want to make, I really need an
> automated test suite that can verify that I'm not breaking anything.
>
> I've just pushed the beginnings of one to the master branch at
> cworth.org:/git/apitrace (commit d24193487b79672c2988e81b07).

Yes, having a test suite around for this pays off.

I'd prefer to have these tests next to the other tests in
https://github.com/apitrace/apitrace-tests .  Having test on a
separate repos is better, as it allows to bisect without changing the
tests in the process.

Note that the test suite needs to stay as portable as apitrace. If you
could rewrite the bash scripts in Python it would be great. If not,
then please make sure that they are only executed on Unix like OSes.

> I think everything there is reasonable to be merged upstream now,
> (except, perhaps, for the last commit which adds a failing test case for
> the "apitrace trim" work that's not there yet).
>
> Feel free to take a look at ti now, or wait until I've got some more
> interesting trim work on top.

The rest looks good to me. I'll merge it soon.

> And if you've got any feedback or questions on the direction it's going,
> I'd be glad to hear that.

Since we last talked about apitrace trim, two ideas crossed my mind:

- instead of an iterative approach (pass over the trace repeatedly
until we converge in a list of calls), do the depency tracking in
precisely 2 passes, 3 steps: 1) scan through the calls maintaining
dependency chains, 2) produce a list of calls on which the call of
interest depends, 3) trim precisely those calls
  - this implies maintaining in every single state object lists of all
calls that define its state, e.g.:
    - framebuffer object
      - list of calls that defined the framebuffer object
      - list of calls that drew into the framebuffer
    - texture object
      - list of calls that define the texture object morphology
      - list of calls for each face/level that define the contents
    - shader object
      - list of calls that define the shader
      - list of calls that define the shader uniforms
    - context:
       - all previous glX* calls
       - all calls that bound the current state

    - a draw call would append the all calls that define the current
context to the framebuffer object dependencies
    - a glClear would clear the dependencies of all previous draw
calls of the current framebuffer object
    - a glTexImage would replace all previous gl(sub)TexImage calls on
the same face/level

    - etc.

- I think it may be easier to have the dependency tracking written in Python:
  - slightly slower runtime, but probably pays up in development time
many times over
    - auto trimming hopefully will be done only once per trace, so it
is ok to be a bit slower
    - using python to process traces is much faster now with "apitrace
pickle" dumper; see scripts/unpickle.py and  scripts/tracediff2.py as
example
  - by doing in it python you have easy access to all information in
specs/*, in particular specs/glapi.py and sideeffects flags;
  - essentially this auxiliary python script would produce the list of
calls, to be trimmed with the pure C apitrace trim command
    - but it can still be called internally by apitrace trim without
the user even be aware of its existence

Anyway, this is jut food for thought. Feel free to do what you think is best.

Jose