tracing on demand

[José Fonseca]

On Mon, Mar 25, 2013 at 10:49 PM, Peter Lohrmann
<peterl at valvesoftware.com> wrote:
>
> I've spent the past week getting up to speed on apitrace, mostly looking into the trace implementation and getting it to trace API calls for a single frame, specified by frame number on the CLI. On its own I don't think this is particularly interesting (and the same result can already be obtained by a full trace & trim to single frame), but there were some bugs in the code generation that I had to fix if tracing is not enabled (look at wglGetProcAddress not exposing the GREMEDY entrypoints and glGetStringi not callling glGetStringi_override(), as two examples).
>
> I'm now planning to implement the tracing on demand, which I saw requested here: https://github.com/apitrace/apitrace/issues/31 My ultimate goal is to allow for a keypress to snapshot the state at the start of the next frame, and then record that frame's API calls. Initially this is just to capture a single frame which can then be replayed, but that can easily be extended to multiple frames in the future. I have implemented similar functionality in the past, so it won't be a difficult feat, but will take time.


One thing to be aware is that not all OpenGL can be queried after it
has been set. For example, if an app uses glCreateShaderProgramv, the
shader source can't be recovered. We have workarounds in glretrace for
that so that we can dump state later (e.g.
https://github.com/apitrace/apitrace/commit/3db3b2f03af32da02faa59b9bcbdd0e2879e280a
). You could mimic such workarounds before tracing starts, or you
could unconditionally trace such calls immediately even before the
trace starts. In particular I suspect that most GLX initialization
calls should be invariably recorded, as I don't think there is any way
to enumerate all contexts, or determine whether they share any state,
and they are very infrequent.

I'm not sure if it's relevant for you, but OpenGL ES is even more
problematic -- one basically needs to shadow a lot of state because
ARB left out major chunks of functionality.

> I don't plan to track state throughout the applications execution, which you have suggested would be a hard task (and I agree), but instead to take a thorough snapshot of all state and buffers and then generate API calls to reproduce that state.

Concerning the emission of the calls, how exactly do envision to
implement it? I suppose that JSON would be unnecessary here, but there
are several helper functions in retrace/glstate* which would probably
be useful for you to extract the state, so we should consider moving
that code to a separate library and subdir so it can be reused for
trace as well, and refactor its implementations to expose more
functionality independent of JSON output.

> As part of this implementation, distinguishing between real calls and fake ones to replay correctly (https://github.com/apitrace/apitrace/issues/59 ) can be resolved by wrapping the fake calls in a documented string marker. I don't know how many of those calls exist currently, but there will definitely be some to replicate the state snapshot at the start of a single-frame  trace.


There several places where we currently emit fake calls (e.g, memcpy,
glVertexAttribPointer*) I actually planned to add a new "flag"
property to call events. The flags already exists (see CALL_FLAG_FAKE
in trace_model.hpp) but I never actually added the flag to the trace
file format. It's an easy thing to do, so I'd suggest you ignore the
fake calls business for now, and I'll take on to myself to finish the
implementation in the near term.

>
> Let me know if you have any suggestions or concerns based on the above,
>
>    Peter


Overall sounds good. It's a great technical challenge, but with
equally great usefulness. Hopefully one can strike some middle ground
where most apps  work well. We could also enhance apitrace to produce
helpful messages when unsupported corner cases are detected, so that
app developers known when and how to avoid them.

Jose