WIP: "apitrace trim" is starting to get interesting

[José Fonseca]

On Mon, Nov 21, 2011 at 11:07 PM, Carl Worth <cworth at cworth.org> wrote:
> I just pushed a couple of new commits to my apitrace-trim branch. In
> addition to trimming drawing operations outside the specified range, the
> new functionality includes:
>
>  * Identifying "dead" textures (never bound within the specified range)
>    and dropping many texture updates to these textures (glTexImage* and
>    friends).

Note that you trim draw calls outside of range, but draw calls to
fbo's to textures later used in the range, need to be preserved as
well.

And so on. That is, multiple passes over the trace are necessary to
get all the needed calls correctly.

Some apps (e.g., wine) draws _everything_ to a fbo (to match dx9
rasterizaion rules I understand) , and the only operation on the
window drawable is a blit. This is probably the toughest nut to crack
for apitrace trim.

>  * Optionally dropping glGet*, glXGet*, and eglGet* calls (only if the
>    user specifies "apitrace trim --trim-gets".

Note that some Get* calls are crucial. Such as:
- GetTexImage, etc. w/ PBO bound
- GetXxxLocation, to allow for attribute location / uniform location remapping

Run

   grep glGet specs/glapi.py | grep -v sideeffects

to see them all. (Hmm. Actually some should not be there).

Anyway, my point is that a regex is no good. A manually written list
will be needed, preferably derived from specs/glapi.py

> Feel free to look at the branch, and let me know what you think.

I confess that I was very sceptical about any state tracking, but
looking at what you done so far, I'm more optimistic that we can trim
a big portion of the cases we care correctly.

> I am not proposing this branch to be merged into master at this
> point.

I'll probably start cherry-picking some of changes.

> Here's an incomplete list of things that need to be addressed
> before the code might be ready for that:
>
>  * The ad-hoc GL-state-tracking code needs to be moved outside of
>    cli_trim.cpp, (perhaps to common/glstatetracker.cpp or so)? This
>    will allow for sharing of frame-detection and similar functionality
>    between trimming and snapshot-generating code, etc. (It will also
>    hide away any of the current, hideously ugly STL usage from the
>    logic doing the actual trimming.)

Agree.

>  * The tracking of live/dead textures needs to be augmented to mark
>    textures live in response to glFramebufferTexture and
>    glFramebufferTextureLayer. This is a correctness fix, (trimming may
>    be incorrectly aggressive without this fix).
>
>  * The list of texture-updating functions (to be potentially trimmed)
>    needs to be made more complete. This is a quality-of-implementation
>    fix, (without this, the tool won't trim as much as possible, but
>    should not generate a buggy trace).

>  * We should decide a better interface than --trim-gets for determining
>    what the user actually wants to trim.

> On this last topic, Eric mentioned to me verbally that he would really
> like to do some fairly fine-grained trimming, (verifying that a trace
> maintains some buggy behavior as each new item is trimmed). For example,
> he would like to be able to trim out-of-range drawing operations first,
> (without touching textures), and only after verifying that trimmed trace
> would he then want to trim dead textures.

> To support something like that, I'm thinking we'll want something like a
> --trim-spec option that might have a default value of:
>
>        "drawing_operations,dead_textures"
>
> and then the user could specify an alternate trim specification, (for
> example, adding "gets" to the list). We might also want separate
> command-line options to add or remove specific classes from the trim
> specification, (since we anticipate the default trim specification to
> grow going forward, and it would be annoying to have to query and enter
> the whole string just to tweak one piece of it).

What I think it would be super is to have a fuzzy trimming couple with
brute force interactive triming:

- a state tracking parser would assign a confidence between 0 and 1
level for each call (0 == must not be trimmed , 1.0 == can be safely
trimmed, everything else an empirical value in between)
- trim all calls with confidence level 1
- verify that a reference frame is preserved (failure would be a bug some where)
- try to trim some of the higher confidence levels
- if reference frame is preserver proceed with remaining calls, other
wise, take a subset of the calls and keep retrying

Another way of tackling this is to have apitrace trim to produce a
list of calls to trim (and not trim), and then let an human manually
edit that list (kind of git rebase -i)

> And of course, once all of the above is done, there are lots of new
> classes of trimming that can be done. For example, it's currently fairly
> disappointing to try trim a trace down to a single frame of calls, but
> still find in qapitrace that every original frame is still present. (We
> might just consider glXSwapBuffers a drawing operation to allow those
> out-of-range frames to disappear.)
>
> It is nice, though, that the current approach allows us to incrementally
> approach the ideal trim-trace-to-a-single-frame operation.
>
> As always, I'll be glad for any feedback. More code to come soon...

This is great stuff!

Once, sorry for my slow responses, but please do not let that deter you.

Jose