Pull request: Dependency-based trim

[José Fonseca]

Hi Carl,

Sorry for not replying sooner. I only had time to glance at your
pataches so far.

I don't object merging this, even if it's not perfect. Just two requests:
- move all trim dependency tracking to a separate module (e.g.,
cli_trim_foo.cpp)
- issue a warning that dependency tracking is experimental and might
not work in all cases

I'll look into more detail as time permits.  No need to post the
series to mailing list -- your repos is fine.

Jose

On Fri, Aug 17, 2012 at 5:23 PM, Carl Worth <cworth at cworth.org> wrote:
> Carl Worth <cworth at cworth.org> writes:
>> So far, my tests pass, but things don't yet quite work when trimming
>> non-trivial traces generated from real-world programs. I think the bugs
>> there are not major and I'll be working on those soon.
>
> I've now fixed several bugs, added several features, and stress-tested
> this with some non-trivial, real-world programs (trimming individual
> frames for hundreds of frames from a few games).
>
> I won't claim that the trim code is perfect yet, (that's an ideal that
> we'll likely never reach, and will require long-term testing), but I do
> think it's ready to be merged upstream, and for people to start using in
> earnest.
>
> Anyone interested, please test this out and report any problems to
> me. I'd love to see traces where trimming any single frame generates a
> trace that draws the wrong thing.
>
> Here are some of the highlights of new things since my last version:
>
>   Better trimming:
>
>     * Lots of fixes for inter-frame texture dependencies
>
>     * Support for multi-texturing, (previously calls to glActiveTexture
>       would confuse the texture tracking).
>
>     * Conservative support to make shaders depend on textures,
>       (previously there were bugs where trim would drop a texture, not
>       realizing that a shader was sampling it).
>
>     * Conservative support for display lists, (previously the trim code
>       ignored display lists entirely and incorrectly resolved all calls
>       immediately). In this version calls within display lists are never
>       trimmed, (and textures bound within display lists are also
>       retained). In a future version we can track display lists with the
>       same resource-tracking framework used for textures to be able to
>       trim unused display lists.
>
>   User-interface improvements:
>
>     * A new --frames option for trimming based on frame numbers rather
>       than call numbers with --calls.
>
>     * A new --print-callset option to find out what trim is actually
>       doing, (and then the option of tweaking that by editing the result
>       and passing it back with --exact --calls=<callset>).
>
>     * Can now quite from "glretrace -w" with 'Q' or 'ESC'.
>
>   More testing framework:
>
>     * There's now a trim_stress directory in apitrace-tests where the
>       user can just drop in a trace file and the test suite will trim
>       out each individual frame and ensure the trimmed traces still draw
>       the same results.
>
> The code is on the master branches of my repositories:
>
>         git://git.cworth.org/git/apitrace
>         git://git.cworth.org/git/apitrace-tests
>
> I've also pushed a "trim-for-upstream" branch to each repository (which
> is currently identical to master). This is just in case I start doing
> something more experimental with master in the future, (though I don't
> have any immediate plans to do so).
>
> José, if you'd like me to post patches on the list in addition to saying
> "go find them in my repository", I'll be happy to. The current series is
> 25 commits to apitrace and 23 commits to apitrace-tests.
>
> From here, there is a bunch of optimization that can be done. We can
> trim out a lot of things that we aren't trimming today, (display lists,
> as mentioned above, lots of redundant state manipulations, buffer
> objects, etc. etc.). It would be useful for me to know what the most
> important things are to be trimmed out next. So, users, let me know what
> you want.
>
> I'm also thinking of doing some performance optimizations soon. A trim
> operations should be relatively rare in the general use case, so it's
> not extremely performance critical. But already in the stress testing in
> the test suite it would be nice if it were much faster. And there's
> definitely a lot of room for improvement here, (the use of
> std::set<unsigned> for both state tracking and the final list of calls
> is currently killing things).
>
> -Carl
>
> _______________________________________________
> apitrace mailing list
> apitrace at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/apitrace
>