[Libreoffice] RFC: Idea for fuzz-testing filters

[Marc-André Laverdière]

Thanks for your feedback.

I would really really like #1, but I'm not knowledgeable enough right
now to do that. If (God willing), I'm starting that PhD soon, I might
just be able to do that blindfolded a year from now :)

For #2, I would like to have a tool generate the format handled by
reading our source code. That would give something like 90% of the spec
generated for us. Then we can fill in the blanks. I'm guessing I could
do that in 2-3 more years of PhD than #1 ;)

As for #3, I haven't had good experience with zzuf. I used the
integrated option from CERT, only to have a few results in the log file
and no fuzzed file to repeat the test with.

I also used zzuf to generate test cases for the wmf filters, put the
results in indeterminate, and then export the VALGRIND variable and run
make -sr. Once that was done, I just had to read the report file for
valgrind errors. Not automated, but 'good enough'.

Problem is that it would saturate the disks pretty fast... so the
end-result is that I was spending more time babysitting The Monster (the
computer which was doing all those computations) than improving stuff.
Not ideal. And I'm talking about >30 Gb disk space filled up! And my
personal beef with zzuf is that the bytes are fuzzed randomly, which
means that you never know if that length field at offset 0xABCDEF was
even touched.

Running the whole of LO on a test case is going to take an enormous
amount of time. There is also the risk of having false positives for
things that would be slow to open (you have to set a time out, after
which it will kill the process).

I think we could have some zzuf going, because the probabilistic thing
could help find the kind of bugs that would be otherwise too expensive
to find deterministically (100 Kb file means a lot of combinations).
Maybe my suggestion could be used to generate a lot of seeds for zzuf.
It would guarantee that the specific byte was tampered with...

I'm not thrilled with the idea of so much process creation and overhead
(think Valgrind) for running a somewhat short test over and over again.

Marc-André Laverdière
Software Security Scientist
Innovation Labs, Tata Consultancy Services
Hyderabad, India

On 10/06/2011 12:01 PM, Huzaifa Sidhpurwala wrote:
> On 10/05/2011 06:41 PM, Caolán McNamara wrote:
>> caolanm->huzaifas: any advice ?
> 
> Nice to see the work you have been doing here!
> 
> To share some opinion about the my work which lead me to the discovery
> of CVE-2011-2713.
> 
> 1. There is no right or wrong approach here. A good approach would be
> the one which covers all the possible code paths or maximum possible
> ones in this case.
> 
> 2. Ideally Peachfuzz or any other intelligent fuzzers (ones available
> freely or custom ones) would be the best way to find flaws. But i will
> have to agree with you, the specs are too big in this case and the time
> taken to translate them into a fuzzer format is formidable.
> 
> 3. I was pointed at [1] by Caolan. How do you run these files through
> libreoffice after generating these test cases?. zzuf could actually
> create the test cases, run libreoffice, destroy them and cycle through
> this process as many times as you want. Saving on hard disk space? :)
> 
>