How can I limit the length of a token in antlr3?

Question

I am trying to build a compiler using antlr and for now I want to limit the length of the identifiers in my language to less than 9.

My code now look like this:

IDENTIFIER: CHAR(CHAR|INT)*;

where CHAR and INT are both fragments. I am wondering if there is any convenient way for me to achieve my goal instead of using this:

IDENTIFIER: CHAR(CHAR|INT)(CHAR|INT)...(CHAR|INT);//repeate (CHAR|INT) 8 times.

Thanks for help.

You might want to have a look at [this SO](http://stackoverflow.com/questions/3056441/what-is-a-semantic-predicate-in-antlr3) question. There is an example on Semantic Predicates that has an example that's similar to what you're asking. — nvlass, Apr 02 '13 at 09:01

score 1 · Answer 1 · answered Apr 02 '13 at 13:49

You should to implement this using a separate check after the lexer is complete. If you attempt to validate the length of your identifier inside the lexer, then input containing Identifier2Long will likely do the following:

Fail to parse Identifier and recover by discarding the I.
Fail to parse dentifier2 and recover by discarding the d.
...more of this
Finally, successfully parse fier2Long as an identifier and return that as the next token.

You could implement the check by overriding Lexer.nextToken.

How can I limit the length of a token in antlr3?

1 Answers1