0

My grammar must ignore whitespaces for the most part, except in certain contexts. Answers to this question advises to define specific lexer rules to handle the exceptions I want.

The problem is that (I think) I cannot handle such cases at the lexer level, since they seem to be triggered high at the parser level.

To be more specific: I want to recognize something like

MyRule:
   MyParseTree1 Operator MyParseTree2 // WS is skipped
   | MyParseTree1 WS SensitiveOperator WS MyParse // WS carries meaning

keeping in mind that I have a WS -> skip rule because in most of my grammar whitespaces should be skipped.

In Xtext, rules can specify on a rule-scoped basis what hidden tokens apply within the rule's scope:

MyRule (hidden COMMENTS):
  ... // WS reaches the parser, comments don't

MyRule2 (hidden WS, COMMENTS):
  ... // WS is skipped, comments too

But I am clueless with antlr4.

AmazingWouldBeGreatBut
  • 65
  • 10
  • Can't you also match the space(s) in your SensitiveOperator? – Bart Kiers Sep 11 '17 at 21:49
  • XText is a paved over version of Antlr3. Lots of concepts don't apply to Antlr4. In any event, looks like you are trying to solve a lexer issue in the parser. Provide the actual lexer rule for `SensitiveOperator`. Confirm that the real issue is that it requires leading and trailing `WS`. – GRosenberg Sep 12 '17 at 02:13
  • Thanks a lot for the answers. I haven't had the time to check your suggestions yet, but I believe defining operators as blank-aware at the lexer level is the way to go. As soon as I check it, I will come back and close the question. – AmazingWouldBeGreatBut Sep 18 '17 at 09:04

1 Answers1

2

If you want to skip certain Tokens depending on the grammar context you should have a look at this question here where the given procedure is described for skipping whitespace (as you want to do it).

Raven
  • 2,951
  • 2
  • 26
  • 42