gui-unbuffered-retrace (Was: Capture and display frame thumbnails in qapitrace.)

[José Fonseca]

On Sat, Mar 24, 2012 at 11:53 AM, José Fonseca <jose.r.fonseca at gmail.com> wrote:
> On Thu, Mar 22, 2012 at 9:12 PM, Dan McCabe <zen3d.linux at gmail.com> wrote:
>> On 03/22/2012 11:11 AM, José Fonseca wrote:
>>> I'm however seeing insane memory usage for large traces. This needs
>>> more investigation. But I think that I'l make thumbnailing
>>> non-automatic (e.g., menu option) until we address this. This would
>>> allow to get this commited in master without causing regressions for
>>> the current use cases, and allow to tackle this later (and turn back
>>> on by default then).
>> The huge memory usage is definitely attributable to thumbnail collection.
>> Right now, that collection is simplistic (i.e., **ALL** frame thumbnails are
>> captured).
>
> I suspect the problem is not the collection of all frame thumbnails,
> but rather QProcess buffering all output (i.e., all uncompressed
> frames) until the process finishes. I think the solution is to
> refactor Retracer and RetraceProcess classes so that the stdout is
> processed while the process is live, and not after.
>
> I've been reading about QThread, QProcess but I'm not yet confident
> enough of the correct way of doing this without inadvertedly
> introducing race conditions or Qt threading rule violations...

I found a solution for this, by eliminating the QThread event loop,
and workaround many quirks in QProcess. I need to do more
testing/benchmarking, but this is now in the gui-unbuffered-retrace
branch.

Zack, if you could share your thoughts on it I'd appreciate it.

I've also noticed that QJSON abuses QIODevice::peek instead of
maintaining a lookahead character, which seems far from efficient,
especially for string processing. It's probably not too hard to fix,
but I rather replace JSON with the trace binary format (or maybe an
uncompressed variant).

Jose