[Libreoffice] [PATCH] Fix for #i93300#: Crash when inserting 32 bit TIFF image

[Harri Pitkänen]

The sample documents in the bug had two problems: 32 bit sample size
and use of floating point sample format[1]. Neither is supported so this
patch adds checks to reject images that have either of these properties.

Additionally a sanity check is added to make sure that similar crash
(division by zero) cannot be induced by creating a malformed image that
specifies the same values for MaxSampleValue and MinSampleValue.

Some literals have been changed from 1 to 1UL since the result will
be assigned to a variable with ULONG data type. With this change it
was actually possible to load the 32 bit image on a 64 bit system but
the colors were distorted (probably due to unsupported sample format).
So rejecting all 32 bit images still seems to be necessary.

This patch has been tested on a 64 bit Linux system using the samples
in the bug and some images from libtiff sample collection at
ftp://ftp.remotesensing.org/pub/libtiff/pics-3.8.0.tar.gz

[1] http://www.awaresystems.be/imaging/tiff/tifftags/sampleformat.html

Signed-off-by: Harri Pitkänen <hatapitk at iki.fi>
---
 filter/source/graphicfilter/itiff/itiff.cxx |   18 +++++++++++++++---
 1 files changed, 15 insertions(+), 3 deletions(-)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Fix-for-i93300-Crash-when-inserting-32-bit-TIFF-imag.patch
Type: text/x-patch
Size: 2487 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/libreoffice/attachments/20101019/e0e38efd/attachment.bin>