1

I'm trying to figure out just how exactly to use ANTLR, but I'm having a really difficult time digesting the things I've found. So far, here are my resources:

A little bit of background

I'm experimenting with moving from JavaParse to ANTLR because I want to handle ASTs of languages besides Java. My understanding of ANTLR and the predefined grammars (linked above) is that this is feasible.

Setup

I created a very simple and standard gradle project in IntelliJ and I'm still encountering issues:

The problem

I'm missing the Java8Lexer and Java8Parser classes. I have no idea where to find these.

build.gradle

group 'com.antlr-demo'
version '1.0-SNAPSHOT'

apply plugin: 'java'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    compile 'org.antlr:antlr4-master:4.5'
    testCompile group: 'junit', name: 'junit', version: '4.11'
}

Test.java

Even in this very trivialized example, none of the two classes I need are not being imported.

public static void parseFile(String f) { // found in Test.java:257
    try {
        if ( !quiet ) System.err.println(f);
        // Create a scanner that reads from the input stream passed to us
        Lexer lexer = new Java8Lexer(new ANTLRFileStream(f)); // missing

        CommonTokenStream tokens = new CommonTokenStream(lexer);

        // Create a parser that reads from the scanner
        Java8Parser parser = new Java8Parser(tokens); // missing
        if ( diag ) parser.addErrorListener(new DiagnosticErrorListener());
        if ( bail ) parser.setErrorHandler(new BailErrorStrategy());
        if ( SLL ) parser.getInterpreter().setPredictionMode(PredictionMode.SLL);

        // start parsing at the compilationUnit rule
        ParserRuleContext t = parser.compilationUnit();
        if ( notree ) parser.setBuildParseTree(false);
        if ( gui ) t.inspect(parser);
        if ( printTree ) System.out.println(t.toStringTree(parser));
    }
    catch (Exception e) {
        System.err.println("parser exception: "+e);
        e.printStackTrace();   // so we can get stack trace
    }
}

It's not described in the pom file...

Community
  • 1
  • 1
djthoms
  • 3,026
  • 2
  • 31
  • 56

3 Answers3

4

I have created an open source project on github that does the parser/lexer generation and the AST creation automatically and in-memory. You can find it on github https://github.com/julianthome/inmemantlr.

The code for getting the AST from a JAVA program is pretty simple:

// the ANTLR grammar
File f = new File("src/test/ressources/Java.g4");
// plug the ANTLR grammar in 
GenericParser gp = new GenericParser(f, "Java");
// load the file that we'd like to parse into a String variable
String s = FileUtils.loadFileContent("src/test/ressources/HelloWorld.java");
// this listener will create an AST from the java file    
gp.setListener(new DefaultTreeListener());
// compile Parser/Lexer
gp.compile();
ParserRuleContext ctx = ctx = gp.parse(s);
// get access to AST
Ast ast = dlist.getAst();
// print AST in dot format
System.out.println(ast.toDot());

If you are interested, you could have a closer look at the test cases in the repository.

Stephan
  • 41,764
  • 65
  • 238
  • 329
Julian
  • 1,694
  • 22
  • 29
1

You are missing one step. You got the java8 grammar but you haven't yet created a a parser from it. This usually involves running the antlr4 jar on the grammar file (FAQs and more), which is very simple (example taken from the Getting Started page):

$ antlr4 Hello.g4
$ javac Hello*.java
Mike Lischke
  • 48,925
  • 16
  • 119
  • 181
1

There is a gradle plugin whose name is antlr to translate g4 files into java files. You can get details of antlr plugin at https://docs.gradle.org/current/userguide/antlr_plugin.html

In addition, you can see the real project using gradle + antlr: https://github.com/todylu/xcodeprojectParser

  • the dependency in antlr grammar project : antlr "org.antlr:antlr4:4.5.3"
  • the dependency in demo project : compile "org.antlr:antlr4-runtime:4.5.3"

By the way, It's better to install ANTLR v4 grammar plugin into Intellij Idea to assist you edit g4 file.

Tody.Lu
  • 915
  • 9
  • 24