I am trying to use the JastAdd system with the ANTLR (v4.7.2) parser generator. I have used it with JJTree/JavaCC, in which JJTree automatically builds the AST, but I want to use ANTLR because I find the documentation to be more robust, and the system as a whole to be more robust. However, I can't figure out how to link up the AST generated by ANTLR to JastAdd. I am also quite new to using these ANTLR, JastAdd and ANT, so if I have done or said something stupid, please be patient.
For days I've been searching for examples for using ANTLR with JastAdd but haven't found any. I figure it should be possible, since JastAdd mentions that any Java-based parser generator can be used with it. I already know how to build an AST (from this post: How to create AST with ANTLR4?), and have done so for a very simple grammar to start to develop an understanding. What I don't understand is how to link the AST generated in my ANTLR code to JastAdd.
I have tried looking into how this is done when using JavaCC/JJTree to see if I could figure out how it's done. From what I can tell, the class names of the AST nodes generated by JJTree have to be the same as the names of the nonterminals in the .ast file, which is used by JastAdd to create classes for each type of node. I believe this because under the "Parsing" heading of the README (http://jastadd.org/releases/jastadd2/R20130412/readme.php), JastAdd mentions, "JavaCC and its tree-building extension, JJTree, are used for parsing. They assume top classes called Node and SimpleNode. Normally, JJTree generates its own AST node subclasses, but we "fool" it to use the AST classes generated by JastAdd instead (by generating them before we run JavaCC/JJTree)." So I have given the same names to my classes that represent AST nodes and the nonterminals in the .ast file (except for the nonterminals extending from abstract operand in lines 5-7 of ArithmeticParser.ast, because these are not nodes, but you can not represent a choice between tokens in a .ast file in any other way as far as I can tell. I'm not too sure how to handle this, maybe I need to add more AST classes?). But I can't really tell if there's anything else I need to do to link them up?
Another thing I saw in the README under the "Understanding the Implementation" heading is that JastAdd creates a partial AST from the .ast file, then eventually .jrag aspect files are added in to the AST, and at the end the AST classes are generated. Several things confuse me about this: (1)This mentions nothing about using the AST generated by whatever parser generator that's being used (be it ANTLR, JJTree/JavaCC, etc.), so I don't really understand how this is worked in to JastAdd. (2) If we are generating the AST classes at the end, how is it beginning to build the AST before this?
I also want to use ant to build the project (and don't know what to do to build the project manually). I have looked at the build.xml file of the tree-building example (http://jastadd.cs.lth.se/examples/CalcTree/index.html), and tried to adapt it to use with ANTLR, but I've been unsuccessful. I get the following error: /Users/Blake/Documents/Dataview_Files/GrammarExANTLR/build.xml:34: Unable to determine generated class.
I have read what little I have been able to find about this online (Why does my ANTLR build Ant task fail with "Unable to determine generated class"?) and (http://palove.kadeco.sk/itblog/posts/40). In the first link, there is a comment:
"If you do a ant -diagnostics on your shell, there should only be ant-antlr.jar and ant-antlr3.jar under ANT_HOME/lib jar listing. If there's also a antlr2?.jar, try removing it."
I did this and I only have ant-antlr.jar, no ant-antlr2.jar, ant-antlr3.jar, or ant-antlr4.jar. Does anyone know where I can get ant-antlr4.jar (if it even exists)? I haven't been able to build the project yet, which further prevents me from digging in to linking my AST in to JastAdd.
So to sum it up, my questions are:
How can I use my AST generated with ANTLR with JastAdd?
How can I use ant with ANTLR?
Are there any examples of using ANTLR with JastAdd?
I'm not sure what code will be necessary, so Ill post a few things and add whatever else may be needed.
The grammar (ArithmeticParser.g4):
/** TOKENS - name must begin with uppercase letter */
// must wrap in () to use -> skip command
WHITESPACE : (' ' | '\t' | '\n' | '\r') -> skip;
EQ : 'equals' | '=' | '->';
OPERATOR : PLUS | MINUS | MULT | DIV | MOD | EXP ;
// these are fragments, meaning they can only be used by LEXER not PARSER
fragment PLUS : 'plus' | '+';
fragment MINUS : 'minus' | '-';
fragment MULT : 'times' | '*';
fragment DIV : 'divby' | '/';
fragment MOD : 'mod' | '%';
fragment EXP : 'pow' | '^';
INT : DIGIT;
fragment DIGIT : [0-9]+;
ID : [a-zA-Z] IDTAIL*;
fragment IDTAIL : [a-zA-Z] | [0-9];
/** Production rules - name must begin with lowercase letter */
root : '<BEGIN' statement+ 'END>' EOF;
statement : (assignment | arithmeticExpression) ';';
assignment : ID EQ arithmeticExpression;
arithmeticExpression : lOp = (INT | ID) OPERATOR rOp = (INT | ID);
ArithmeticParser.ast
RootNode ::= StatementNode*;
abstract StatementNode;
AssignmentNode:StatementNode ::= <ID> ArithmeticExpressionNode;
ArithmeticExpressionNode:StatementNode ::= lOp:operand <OPERATOR> rOp:operand;
abstract operand;
intOp:operand ::= <INT>;
idOp:operand ::= <ID>;
The classes corresponding to node types:
/**
* Required as superclass of all nodes for AstBuilderVisitor class
*/
abstract class AstNode { }
class RootNode extends AstNode {
// children nodes of the root node
ArrayList<StatementNode> statementList;
public RootNode() {
this.statementList = new ArrayList<>();
System.out.println("\tNEW RootNode successfully created");
}
}
abstract class StatementNode extends AstNode { }
class AssignmentNode extends StatementNode {
String id;
ArithmeticExpressionNode arithmeticExpression;
AssignmentNode(String id, ArithmeticExpressionNode arithmeticExpression) {
this.id = id;
this.arithmeticExpression = arithmeticExpression;
System.out.println("\tNEW AssignmentNode: id = " + id);
}
}
class ArithmeticExpressionNode extends StatementNode {
char operator;
char lOpType, rOpType;
String lOp, rOp;
public ArithmeticExpressionNode(char operator, char lOpType, char rOpType, String lOp, String rOp) {
this.operator = operator;
this.lOpType = lOpType;
this.rOpType = rOpType;
this.lOp = lOp;
this.rOp = rOp;
System.out.println("\tNEW ArithmeticExpressionNode: " + lOp + " " + operator + " " + rOp);
}
}
And finally the build.xml file:
<!-- A project is one or more targets that executes tasks. This project
has three targets: build, clean, and test. The build target is
set as the default target and used if no target is supplied. -->
<project name="Compiler" default="build" basedir=".">
<!-- The name used for the ANTLR files -->
<property name="parser.name" value="ArithmeticParser"/>
<!-- The directory where generated files will be stored -->
<property name="package" value="AST"/>
<!-- The directory where tools like ANTLR and jastadd are stored. -->
<property name="tools" value="tools"/>
<!-- The JastAdd ANT task -->
<taskdef classname="jastadd.JastAddTask" name="jastadd" classpath="${tools}/jastadd2.jar" />
<!-- gen:
- Creates a directory for generated files.
- Generates the AST classes using jastadd.
- Generates the parser using antlr. -->
<target name="gen">
<mkdir dir="${package}"/>
<jastadd package="${package}" jjtree="false" grammar="${parser.name}">
<fileset dir=".">
<include name="**/*.ast"/>
<include name="**/*.jrag"/>
<include name="**/*.jadd"/>
</fileset>
</jastadd>
<antlr
target="${parser.name}.g4"
outputdirectory="${package}"
/>
</target>
<!-- build: (automatically runs "gen" if needed)
- compiles all java files
- intended to be used from the command line
(in Eclipse you don't need this target since Eclipse compiles
java files automatically)
- you can change "javac1.4" to "jikes" for faster compilation -->
<target name="build" depends="gen">
<javac
compiler="javac1.4"
srcdir="."
classpath="${tools}/junit.jar"
/>
</target>
<!-- clean:
- deletes the directory holding generated files
- deletes all .class files (recursively) -->
<target name="clean">
<delete dir="${package}"/>
<delete>
<fileset dir="." includes="**/*.class"/>
</delete>
</target>
</project>
EDIT: Ok, I found a way to do it so that in build.xml I replaced
<antlr
target="${parser.name}.g4"
outputdirectory="${package}"
/>
with
<java jar="tools/antlr-4.7.2-complete.jar" fork="true">
<arg value="-o"/>
<arg value="."/>
<arg value="${parser.name}.g4"/>
</java>
so now antlr runs. But now JastAdd doesn't run at all. I'm not sure why this is. It would still be useful to have ant-antlr4.jar if that exists somewhere. I wonder if just using this <java> ant task is causing JastAdd not to run for some reason?
