Antlr grammar, implicit token definition in parser rule

Question

A weird thing is going on. I defined the grammar and this is an excerpt.

name  
   : Letter                 
   | Digit name              
   | Letter name          
   ;

numeral  
   : Digit                
   | Digit numeral         
   ;

fragment  
Digit  
   : [0-9]  
   ;  

fragment  
Letter  
   : [a-zA-Z]  
   ;

So why does it show warnings for just two lines (Letter and Digit name) where i referenced a fragment and others below are completely fine...

just one warning(125): implicit definition of token 'Letter' in parser ..... and then for the digit in Digit name line it says the same with 'Digit'.. Implicit token definition in parser rule — Sava Gavran, Jul 21 '14 at 10:20
So i just erased the fragment lines and there are no errors. Is this ok, will it do the right thing now? — Sava Gavran, Jul 21 '14 at 10:30

score 8 · Answer 1 · edited May 23 '17 at 12:00

8

Lexer rules you mark as fragments can only be used by other lexer rules, not by parser rules. Fragment rules never become a token of their own.

Be sure you understand the difference: What does "fragment" mean in ANTLR?

EDIT

Also, I now see that you're doing too much in the parser. The rules name and numeral should really be a lexer rule:

Name
 : ( Digit | Letter)* Letter
 ;

Numeral
 : Digit+
 ;

in which case you don't need to account for a Space rule in any of your parser rules (this is about your last question which was just removed).

edited May 23 '17 at 12:00

Community

1
1

answered Jul 21 '14 at 10:23

Bart Kiers

166,582
36
299
288

I used the C grammar as an example. Here : https://github.com/gavransava/grammars-v4/commit/15df18159653f7ae2963cc73bec7d2b8939a9363#diff-74edde23ba1382b96fb11735598f5dacR503 ... And you are right, there is an Identifier rule which is not a fragment, it is used in parser rules, but is not a fragment itself. But it still contains fragment in its own definition.. – Sava Gavran Jul 21 '14 at 10:26
@SavaGavran, also check my **EDIT**. – Bart Kiers Jul 21 '14 at 20:19

john k · Answer 2 · 2020-07-29T14:21:09.120

0

Just in case you are using an older version of antlr: [0-9]

and

[a-zA-Z]

are not valid regular expressions in old Antlr. replace them with

'0'..'9'

and

('a'..'z' | 'A'..'Z')

and your issues should go away.

edited Jul 29 '20 at 14:21

answered Jan 30 '18 at 04:33

john k

6,268
4
55
59

1

In ANTLRv4 now they are valid expressions, see https://github.com/antlr/grammars-v4/blob/master/java/JavaLexer.g4#L207 – Matthias B Jul 03 '19 at 14:21

Antlr grammar, implicit token definition in parser rule

2 Answers2

EDIT