3

With ANTLR2, you could define something like this in grammar definition file:

options
{
   language = "CSharp";
   namespace = "Extended.Tokens";
}

tokens {
   TOKEN<AST=Extended.Tokens.TokenNode>;
}

And then, you could create a class:

public class TokenNode: antlr.BaseAST
{
    ...
}

Any ideea if something like this can be used (delegate class creation to AST factory instead of me doing the tree replication manually)? It's not working just by simple grammar definition copy from old to new format, and I tried to search their site and samples for somthing similar. Any hints?

EDIT

I'm not trying to create custom tokens, but custom 'node parsers'.

In order to 'execute' a tree you have 2 choices (as far as I understood):

  1. create a 'tree visitor' and handle values, or
  2. create a tree parser by 'almost-duplicating' the grammar definition.

In v2 case, I could decorate the tree node to whateveer method I would have liked and then call them after the parser ran by just calling something like 'execute' from root node.

Bart Kiers
  • 166,582
  • 36
  • 299
  • 288
dcg
  • 1,144
  • 1
  • 22
  • 38
  • Sorry, It's a typo: this definition was with v2. I edited the post. – dcg Oct 03 '11 at 13:48
  • Does it makes any sense? Is there any way to implement in ANTLR3 something similar to ANTLR2? – dcg Oct 03 '11 at 13:58
  • Not custom tokens, but custom 'node parsers'. In order to 'execute' a tree you have 2 choices (as far as I understood): 1. create a 'tree visitor' and handle values and 2. to create a tree parser by 'almost-duplicating' the grammar definition. In v2 case, I could decorate the tree node to whateveer method I would have liked and then call them after the parser ran by just calling something like 'execute' from root node. – dcg Oct 03 '11 at 15:34

1 Answers1

4

I know little C#, but there shouldn't be much difference with the Java target.

You can create - and let ANTLR use - a custom tree by setting the ASTLabelType in the options { ... } section (an XTree in this case):

T.g

grammar T;

options {
  output=AST;
  ASTLabelType=XTree;
}

tokens {
  ROOT;
}

@parser::header {
  package demo;
  import demo.*;
}

@lexer::header {
  package demo;
  import demo.*;
}

parse
  :  Any* EOF -> ^(ROOT Any*)
  ;

Any
  :  .
  ;

You then create a custom class which extends a CommonTree:

demo/XTree.java

package demo;

import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;

public class XTree extends CommonTree {

  public XTree(Token t) {
    super(t);
  }

  public void x() {
    System.out.println("XTree.text=" + super.getText() + ", children=" + super.getChildCount());
  }
}

and when you create an instance of your TParser, you must create and set a custom TreeAdaptor which creates instances of your XTree:

demo/Main.java

package demo;

import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;

public class Main {

  public static void main(String[] args) throws Exception {
    String source = "ABC";
    TLexer lexer = new TLexer(new ANTLRStringStream(source));
    TParser parser = new TParser(new CommonTokenStream(lexer));
    parser.setTreeAdaptor(new CommonTreeAdaptor(){
      @Override
      public Object create(Token t) {
        return new XTree(t);
      }
    }); 
    XTree root = (XTree)parser.parse().getTree();
    root.x();
  }
}

Running the demo:

java -cp antlr-3.2.jar org.antlr.Tool T.g -o demo/
javac -cp antlr-3.2.jar demo/*.java
java -cp .:antlr-3.2.jar demo.Main

will print:

XTree.text=ROOT, children=3

For more info, see: http://www.antlr.org/wiki/display/ANTLR3/Tree+construction

Bart Kiers
  • 166,582
  • 36
  • 299
  • 288
  • This is not quite I was looking for ... In the sample I saw (http://www.antlr2.org/doc/trees.html - "AST Factories" and "Heterogeneous ASTs") you are able to define what class to create for a specific token. In your sample, you can create a single object type (XTree) after the tree is parsed. – dcg Oct 04 '11 at 07:40
  • Well, in the `create` method of the `CommonTreeAdaptor` you can return any tree you like, based on the token that is passed as a parameter, whether that's an `XTree` or `YTree`, it doesn't matter. In that case, you may want to change `ASTLabelType` to `CommonTree` or some other class that all your custom trees extend. – Bart Kiers Oct 04 '11 at 07:45
  • I think I found what I was looking for: http://www.antlr.org/wiki/display/ANTLR3/Tree+construction (Heterogeneous tree nodes). Thanks for hints! :-) – dcg Oct 04 '11 at 08:11
  • I _just_ posted a link to that! :) You're welcome, of course. – Bart Kiers Oct 04 '11 at 08:13