Mesa (master): glcpp: Rework lexer to use a SKIP state rather than REJECT.

[Kenneth Graunke]

Module: Mesa
Branch: master
Commit: f20656e9446b9acde55b9dd41a2b47d7d5b7a56a
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=f20656e9446b9acde55b9dd41a2b47d7d5b7a56a

Author: Kenneth Graunke <kenneth at whitecape.org>
Date:   Tue Mar  1 12:24:58 2011 -0800

glcpp: Rework lexer to use a SKIP state rather than REJECT.

Previously, the rule deleted by this commit was matched every single
time (being the longest match).  If not skipping, it used REJECT to
continue on to the actual correct rule.

The flex manual advises against using REJECT where possible, as it is
one of the most expensive lexer features.  So using it on every match
seems undesirable. Perhaps more importantly, it made it necessary for
the #if directive rules to contain a look-ahead pattern to make them
as long as the (now deleted) "skip the whole line" rule.

This patch introduces an exclusive start state, SKIP, to avoid REJECTs.
Each time the lexer is called, the code at the top of the rules section
will run, implicitly switching the state to the correct one.

Fixes piglit tests 16384-consecutive-chars.frag and
16385-consecutive-chars.frag.

---

 src/glsl/glcpp/glcpp-lex.l |   37 ++++++++++++++++---------------------
 1 files changed, 16 insertions(+), 21 deletions(-)

diff --git a/src/glsl/glcpp/glcpp-lex.l b/src/glsl/glcpp/glcpp-lex.l
index 11b73ae..2c9eda6 100644
--- a/src/glsl/glcpp/glcpp-lex.l
+++ b/src/glsl/glcpp/glcpp-lex.l
@@ -57,7 +57,7 @@ void glcpp_set_column (int  column_no , yyscan_t yyscanner);
 %option stack
 %option never-interactive
 
-%x DONE COMMENT UNREACHABLE
+%x DONE COMMENT UNREACHABLE SKIP
 
 SPACE		[[:space:]]
 NONSPACE	[^[:space:]]
@@ -74,6 +74,17 @@ OCTAL_INTEGER		0[0-7]*[uU]?
 HEXADECIMAL_INTEGER	0[xX][0-9a-fA-F]+[uU]?
 
 %%
+	/* Implicitly switch between SKIP and INITIAL (non-skipping);
+	 * don't switch if some other state was explicitly set.
+	 */
+	glcpp_parser_t *parser = yyextra;
+	if (YY_START == 0 || YY_START == SKIP) {
+		if (parser->lexing_if || parser->skip_stack == NULL || parser->skip_stack->type == SKIP_NO_SKIP) {
+			BEGIN 0;
+		} else {
+			BEGIN SKIP;
+		}
+	}
 
 	/* Single-line comments */
 "//"[^\n]* {
@@ -137,6 +148,7 @@ HEXADECIMAL_INTEGER	0[xX][0-9a-fA-F]+[uU]?
 	yylineno = strtol(ptr, &ptr, 0) - 1;
 }
 
+<SKIP,INITIAL>{
 {HASH}ifdef/.*\n {
 	yyextra->lexing_if = 1;
 	yyextra->space_tokens = 0;
@@ -170,27 +182,10 @@ HEXADECIMAL_INTEGER	0[xX][0-9a-fA-F]+[uU]?
 	yyextra->space_tokens = 0;
 	return HASH_ENDIF;
 }
-
-	/* When skipping (due to an #if 0 or similar) consume anything
-	 * up to a newline. We do this with less priority than any
-	 * #if-related directive (#if, #elif, #else, #endif), but with
-	 * more priority than any other directive or token to avoid
-	 * any side-effects from skipped content.
-	 *
-	 * We use the lexing_if flag to avoid skipping any part of an
-	 * if conditional expression. */
-[^\n]+/\n {
-	/* Since this rule always matches, YY_USER_ACTION gets called for it,
-	 * wrongly incrementing yycolumn.  We undo that effect here. */
-	yycolumn -= yyleng;
-	if (yyextra->lexing_if ||
-	    yyextra->skip_stack == NULL ||
-	    yyextra->skip_stack->type == SKIP_NO_SKIP)
-	{
-		REJECT;
-	}
 }
 
+<SKIP>[^\n] ;
+
 {HASH}error.* {
 	char *p;
 	for (p = yytext; !isalpha(p[0]); p++); /* skip "  #   " */
@@ -293,7 +288,7 @@ HEXADECIMAL_INTEGER	0[xX][0-9a-fA-F]+[uU]?
 	}
 }
 
-\n {
+<SKIP,INITIAL>\n {
 	yyextra->lexing_if = 0;
 	yylineno++;
 	yycolumn = 0;