[HarfBuzz] Test suite finally coming along

[Behdad Esfahbod]

Just a quick note to say that I just pushed 44 commits to master.  The email
notification is a whopping 1mb, so I'm not going to approve it.

What I've been doing is adding the tools needed to create a nice test suite,
and I'm glad to say that I'm very close to that now.  Here's a short rundown
of how things are supposed to work:

- hb-shape now has options --show-text, --show-unicode, and
--show-line-numbers.  The output then looks something like this:

8: (अँधा)
8: <U+0905,U+0901,U+0927,U+093E>
8: [uni0905=0+1602|uni0901=0 at 1|uni0927=2+1442|uni093E=2+534]

The nice thing about this format is that it gives us all the information we
need to compare two backends (ot and uniscribe for example) and link back the
problem to the original text and location in the test suite.

Oh, BTW, I changed hb-shape and hb-view to take and generate cluster info in
character indices, not byte indices.  This also affects what --features takes.


Moving on to the new test/shaping director:  Under test/shaping you will find
the following tools now:

- hb-diff: This used to be in util.  Basically what it does is like a regular
diff, but compares the corresponding lines of the two files only.  Ie.
compares line #1 to line #1, then line #2 to line #2.  Plus, it colorfully
highlights which parts of the lines are different.  It's very useful visually,
but as we'll see later, we can use the highlight information in the other tools.

Run it like this:

./hb-diff --color output1 output2 | less -r

Then there's:

- hb-diff-filter-failures: this one takes the diff output and only passes out
the failure lines.  Right now it simply does the equivalent of "grep '^[-+]'"
but it will be extended to keep the text and Unicode lines for the failing
tests also (if line numbers are available).

I will add more filters later, for example, some failures may not actually be
failures if the clusters are similar and glyphs get positioned similarly but
eg the advance values within clusters are different.

Another filter I will add soon is, well, summarizing numbers of failures.


Then there are a few Unicode tools:


- hb-unicode-decode:

behdad:shaping 0$ echo अँधा | ./hb-unicode-decode --stdin
<U+0905,U+0901,U+0927,U+093E>
behdad:shaping 0$ ./hb-unicode-decode अँधा
<U+0905,U+0901,U+0927,U+093E>


- hb-unicode-encode:

behdad:shaping 0$ echo "<U+0905,U+0901,U+0927,U+093E>" | ./hb-unicode-encode
--stdin
अँधा
behdad:shaping 0$ ./hb-unicode-encode 905 901,927 U+093e
अँधा


- hb-unicode-prettyname:

behdad:shaping 0$ echo "<U+0905,U+0901,U+0927,U+093E>" |
./hb-unicode-prettyname --stdin
A + CANDRABINDU + DHA + AA-MATRA
behdad:shaping 0$ ./hb-unicode-prettyname 0905 0901 0927 093E
A + CANDRABINDU + DHA + AA-MATRA


So, with these in place we can now store test data in UTF-8 only and generate
the more descriptive forms automatically.


Then we have to manage text files and font files.  I added two tools:

- hb-manifest-update: creates MANIFEST files for an directory and its
subdirectories recursively.  A MANIFEST file simply lists the files and
directories in this directory, but excludes files named README, LICENSE,
COPYRIGHT, SOURCES, AUTHORS, and ChangeLog.

- hb-manifest-read: uses the MANIFEST files to generate a list of all files in
a directory.

So for example now I can do:

behdad:shaping 0$ ./hb-manifest-read
texts/in-tree/shaper-indic/indic/script-devanagari
texts/in-tree/shaper-indic/indic/script-devanagari/misc/misc.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-AdditionalConsonants.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-AdditionalVowels.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-Consonants.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-DependentVowels.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-DevnagariSpecificAddition.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-Digits.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-GenericPunctuation.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-IndependentVowels.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/codepoint/IndicFontFeatureCodepoint-VariousSigns.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/gpos/IndicFontFeatureGPOS-AboveBase.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/gpos/IndicFontFeatureGPOS-BelowBase.txt
texts/in-tree/shaper-indic/indic/script-devanagari/utrrs/gsub/IndicFontFeatureGSUB.txt

I can do the same for a fonts tree too.

With these in place, now I just need a master matching rule file to express
which text files should be tested against which fonts.  To begin with there's
really just one rule: test any text file with "script-*" with fonts that have
the same "script-*" in their path.  Then we can add a tool to get the texts
and fonts manifests, the matching rules, and generate a list of text,font
pairs to be tested.  Those then can be tested in parallel and outputs stored,
or compared directly and the diffs stored.

When we have all the diffs (only the failures perhaps), it's a matter of
loading them and summarize or visualize them.

I currently am planning to use the text,font pair to generate a hash to store
the output / diff for this pair.  Right now I don't plan to include expected
outputs in-tree, so this is more of a comparative test suite.

Another tool I will add is something like hb-render or something, which would
take hb-shape output and render it.  Essentially, hb-view then would
functionally become "hb-shape | hb-render".  We can actually do better:
hb-render can understand the hb-diff format to mark differing glyphs red/green
to help understand the differences.

Anyway, that's it for now.  I went ahead and imported test data from UTRRS as
well as test-shape-complex into the new framework.  There's no test runner
right now however.  I probably spend more time on this during the week, but
others should feel free to play with it and submit patches!


Cheers,
behdad