[Mesa-dev] [PATCH 0/9][RFC] GLSL preprocessor/parser improvements

[Thomas Helland]

This patch series contains some of the work done by Vladislav
in the beginning of March, that seems to have been forgotten.

For reference, that series, with review comments, can be found here:
https://lists.freedesktop.org/archives/mesa-dev/2017-January/139892.html

I've adressed some of the review comments, most notably redoing the
string buffer implementation quite a bit, and adding tests.
The major change is that it is now decoupled from the parser
and instead lives in /src/util/ as a utility for easier reuse.
I've also closely followed the structure of our hash table and set
so that it should be quite familiar for everyone. My implementation
is also null terminated, while Vladislav's implementation was not.
My reasoning behind this was that it's less fragile if someone where
to access the contents of the buffer directly. I've also expanded
with some different functions that allow us to do more stuff.
Obviously this could be expanded upon further if need be.

I've included some of Vladislav's less involved patches that
I think we should get in as soon as possible. I've added Ian's
r-b on the ones that he reviewed (I hope that's OK). The one
patch that is missing from this series is the hand-written
custom parser. While this was the thing that gave the biggest
speed-up, unfortunately it's also a bit harder to review,
so I've left that as a future exercise.

I've run it through my complete shader-db and there's no changes
or breakages, so that's encouraging. It amounts to about a 3%
reduction in runtime and executed instructions on the shader-db run.
It should be noted that the i965 backend is the primary bottleneck
when running shader-db on my computer, so the numbers don't look all
that impressive due to this. I'll see if I can come up with some
numbers using the gallium noop driver.

I have not done a very thorough job on testing that the output
of the preprocessor/parser is the same after this series.
However, there's no changes here that I believe should cause
any issues, so I feel quite confident that this series should
not cause any trouble. The original series did have some issues
discovered by running a fuzzer over it, however I expect that
was caused by the introduced hand-written fast-path scanner and 
associated macro substitutaion that are not a part of this series.

CC: Vladislav Egorov <vegorov180 at gmail.com>
CC: Ian Romanick <idr at freedesktop.com>

Thomas Helland (5):
  util: Add a string buffer implementation
  util: Add tests for the string buffer
  glsl: Change the parser to use the string buffer
  glcpp: Use string_buffer for line continuation removal
  glcpp: Avoid unnecessary call to strlen

Vladislav Egorov (4):
  glcpp: Avoid unnecessary strcmp()
  glcpp: Skip unnecessary line continuations removal
  ralloc: Use strnlen() inside of strncat()
  glcpp: Use Bloom filter before identifier search

 configure.ac                                    |   1 +
 src/compiler/glsl/glcpp/glcpp-lex.l             |   3 +-
 src/compiler/glsl/glcpp/glcpp-parse.y           | 123 +++++++++-------
 src/compiler/glsl/glcpp/glcpp.h                 |  18 ++-
 src/compiler/glsl/glcpp/pp.c                    |  73 ++++++----
 src/util/Makefile.am                            |   3 +-
 src/util/Makefile.sources                       |   2 +
 src/util/ralloc.c                               |   7 +-
 src/util/string_buffer.c                        | 180 ++++++++++++++++++++++++
 src/util/string_buffer.h                        |  75 ++++++++++
 src/util/tests/string_buffer/Makefile.am        |  34 +++++
 src/util/tests/string_buffer/append_and_print.c |  82 +++++++++++
 12 files changed, 505 insertions(+), 96 deletions(-)
 create mode 100644 src/util/string_buffer.c
 create mode 100644 src/util/string_buffer.h
 create mode 100644 src/util/tests/string_buffer/Makefile.am
 create mode 100644 src/util/tests/string_buffer/append_and_print.c

-- 
2.13.0