0

I'm currently making a RUBY language parser. My question is that I want to use two tokens for one character. For example, in the lexer I write

"(" { return tLPAREN; return tLPAREN2; } while the token tLPAREN2 DON,T WORK.

How to make two tokens can handle 1 character. It would help me get rid of conflicts in grammar.

Flex version 2.6.3 source code and win_bison based on Bison version 2.7

"(" { return tLPAREN; return tLPAREN2; } while the token tLPAREN2 DON,T WORK.

YANIX
  • 3
  • 1
  • You can't. At least not within one rule. Each token can only have one type. If you have conflicts fix your rules so that you can decide whether you have a `tlparen1` or `tlparen2` – derpirscher Jun 11 '23 at 17:49
  • Does this answer your question? [How can flex return multiple terminals at one time](https://stackoverflow.com/questions/42434603/how-can-flex-return-multiple-terminals-at-one-time) – Piotr Siupa Jun 12 '23 at 12:32
  • Although, if you need such tricks to get rid of conflicts in your parser, something may be wrong with how you've written your grammar. – Piotr Siupa Jun 12 '23 at 12:34
  • i control the choice of the ruby ​​parser, in which just two tokens return the same value, such as (returned by the sale of tokens – YANIX Jun 13 '23 at 11:06

1 Answers1

0

You can do something like this with start states. You'll recognize the token and have the action set an (exclusive) start state, back up the input and return the first token. Then in the start state, you'll recognize the token again and go back to the normal start state and return the second token. So something like:

%x Paren2

%%

"("             { BEGIN(Paren2); yyless(0); return tLPAREN; }
<Paren2>"("     { BEGIN(INITIAL); return tLPAREN2; }

Note the call to yyless(0) which "pushes back" the ( to be recognized again by the second rule.

Chris Dodd
  • 119,907
  • 13
  • 134
  • 226
  • Thank you! This is the first time I see this, but I will definitely try to do this option. <3 – YANIX Jun 12 '23 at 09:16