From thomasrinsma at gmail.com  Fri Mar 13 06:04:09 2015
From: thomasrinsma at gmail.com (Thomas Rinsma)
Date: Fri, 13 Mar 2015 14:04:09 +0100
Subject: [fprint] vfs5011 rescale_image MAX_OFFSET increase
Message-ID: <CAGOjmJL0CQ-ETUVqvp1X_TERoErJ+SHXbd2DfGhupdMWsrQUKA@mail.gmail.com>

Hi all,

My laptop contains a Validity Sensors fingerprint reader (138a:0017).
There is support for this reader in libfprint, however I experienced a
bug where the scanned image appeared to be stretched vertically in
large portions. This could only be solved by scanning my finger really
quickly.

The output of fprint_demo seemed ot contain a lot of messages of the form:
fp:debug [vfs5011_rescale_image] offsets: 1426251065051379 - 10

Where the rightmost number was often 10. After having a quick look at
the code (libfprint/drivers/vfs5011.c), I saw that this was the offset
detected by the rescale function, which has a constant MAX_OFFSET
which is set to 10. I think this means that it didn't find the best
match in 10 lines. So I tried to set it to 20, and now everything
works fine. The highest numbers that are now produced are around 18 or
19, which is near 20, so it might have to even be a higher maximum.

To recap, I fixed my problem by changing line 354 of vfs5011.c from:
        MAX_OFFSET = 10,
to
        MAX_OFFSET = 20,

I've not provided a patch because I barely understand this code and 20
is just a magic number that worked for me (double the previous). Hope
this helps anyone else who does understand this a little bit better.

Cheers,
Thomas