0

I am creating a DSL using antlr. Lexer and Parser are written in one grammar file(say layout.g). Tree grammar is written in another grammar file (say layoutTree.g). Now Tree parser is not properly parsing. I printed the AST output from parser, and its correct. I walked through the generated tree parser code, and found that token value declarations assign different values in tree parser and parser.Below is the sample output from parser and tree parser.

Parser output

public static final int ARRAY_MEMBER_TOKEN=4;
public static final int ARRAY_TOKEN=5;
public static final int DECLARATION_TOKEN=6;

Tree Parser Output

public static final int EOF=-1;
public static final int DECLARATION_TOKEN=4;
public static final int IDENTIFIER=5;

As you can see DECLARATION_TOKEN has different value in parser output and tree parser output. Because of this tree parser is not working as expected.How can I correct this problem?

Is it a problem with generated token file(say layout.token)? This file is empty in my project.How can I generate this file?

Bart Kiers
  • 166,582
  • 36
  • 299
  • 288
hnm
  • 789
  • 2
  • 9
  • 24
  • A `.tokens` file is generated automatically while generating your lexer/parser(s). The fact that your `.tokens` file is empty, is not good. How are you generating your lexer/parser(s)? (command line, IDE, ANTLRWorks, other?) – Bart Kiers Jul 25 '11 at 20:26
  • I use 'antlrv3ide' from sourceforge.net. I have installed it from "http://antlrv3ide.sourceforge.net/updates". How can I generate it using command line? – hnm Jul 26 '11 at 01:53
  • 1
    On the command line, do: `java -cp antlr-3.2.jar org.antlr.Tool layout.g` and then `java -cp antlr-3.2.jar org.antlr.Tool layoutTree.g` – Bart Kiers Jul 26 '11 at 06:36

1 Answers1

0

Do you have something like this in your tree grammar? 

options
{
  tokenVocab=layout; //NOT layout.g or layout.tokens
  ASTLabelType=pANTLR3_BASE_TREE;
}
  • I have ASTLabelType=CommonTree; in my parser grammar. – hnm Jul 26 '11 at 01:48
  • sorry I missed tokenVocab=layout; in above comment.I have options options { language = Java; tokenVocab = layout; ASTLabelType = CommonTree; } in tree grammar – hnm Jul 26 '11 at 01:57