Generating ANTLR4 grammar files with package declaration in gradle

Question

I try to use the gradle antlr plugin, but run into several problems.

I have a grammar file called wls.g4:

grammar WlsScript;

@header {
   package hu.pmi.wls.antlr.ws;
} 

program
  : 'statementList'? EOF
  ;

// Several more grammar and lexer rules

(Note: I made the statementList to a keyword only to make a correct grammer without including the whole grammar. ;-))

This file is located in /src/main/antlr (as the default source folder of the antlr production grammar files).

Here is the snippet from build.gradle:

project('common') {

    apply plugin: 'antlr'

    dependencies {
       // Some dependencies

       antlr "org.antlr:antlr4:4.5"
    }
}

When I use the generateGrammarSource gradle task (comming from antlr plugin) to generate the source files it generates the files in build/generated-src/antlr/main folder which is the default.

What goes wrong, that it doesn't create the folders of the java package (hu/pmi/wls/antlr/ws in our case) so the source will be incorrectly located in Eclipse.

My primary question is how could I force the task to generate source files in a package-structured way? In other words, how can I configure the gradle task to use the package declaration from grammar?

Thanks!

I am having the same issue with ANTLR 4.5. I am only using the org.antlr.v4.Tool to process the .g4 file and couldn't get it to generate the Java source in their corresponding package name folder structure. It could generate files according to the directory specified by the "-o" option. So the only way is to specify explicitly the package folder structure in the "-o" option. I am not familiar with Gradle but lookout if there's a way for Gradle to pass in options while executing the ANTLR tool. — anonymous, Jul 17 '15 at 00:37

score 17 · Answer 1 · answered Jul 27 '15 at 07:06

17

I had the exact same question and I was able to change the outputDirectory used by the AntlrTask to get the package name in there.

generateGrammarSource {
    outputDirectory = new File("${project.buildDir}/generated-src/antlr/main/net/example".toString())
}

I found no easy way to extract the grammar package from the files so had to hardcode it in the task.

answered Jul 27 '15 at 07:06

Nagesh Susarla

1,660
10
10

How do you reconcile this in an IDE? I could write something like `import whatever.gen.MyParser` and `MyParser x = new MyParser(...)`, but the ide screams away about how it cant find source for that --because the source is 'just in time', for lack of a better phrase. Maybe its better to have a gen-src folder thats acknowledged by the IDE but ignored under version control? is that antithetical to gradle conventions? – Groostav Mar 30 '16 at 07:37
1

You can add the generated-src directory as a source directory in IDE (I'm using IntelliJ) – Nagesh Susarla Mar 30 '16 at 20:48

score 7 · Answer 2 · answered Feb 14 '17 at 01:56

7

Add this to your sub-project build.gradle

generateGrammarSource {
    outputDirectory = file("src/main/java/com/example/parser")
}

add this to your grammar after your "grammar ";

@header {
    package com.example.parser;
}

Tested and working with Java8 grammar from antlr example grammars

Additional Link(s):

Here is a short guide of the Antlr plugin from docs.gradle.org

I haven't tested this while putting it in parent-project like it looks like you did, but should be easy enough to extend to do that. (I try to keep parent project empty other than repositories to use.)

answered Feb 14 '17 at 01:56

mawalker

2,072
2
22
34

[This](https://stackoverflow.com/questions/36469546/what-is-minimal-sample-gradle-project-for-antlr4-with-antlr-plugin) answer suggests that setting the outputDirectory into your source code like this is dangerous because a gradle 'build clean' can then wipe out *all* source files in that directory. The answer suggests moving the generated files via gradle's 'copy' command to their future locations AFTER they are generated into generated-src. – Some Guy Jan 08 '19 at 02:06

Artur · Answer 3 · 2018-02-28T09:58:59.813

Transfer a generated ANTLR files to the project folder and ZZZ subfolder.

antlr.g4

....

@header {
package XXX.YYY.ZZZ;
}

.....

build.gradle

group 'XXX'
version '0.0.1'

apply plugin: 'java'
apply plugin: 'application'
apply plugin: 'antlr'
apply plugin: 'eclipse'

mainClassName = 'XXX.YYY.Test'
final antlr4Version = '4.7'
final antlrGen = 'gen'
final baseDir = 'src/main/java'

repositories {
    mavenCentral()
}

dependencies {
    antlr "org.antlr:antlr4:"+antlr4Version
}

generateGrammarSource {
    outputDirectory = file(antlrGen)
    maxHeapSize = '64m'
    arguments += ['-visitor', '-long-messages']
}

generateGrammarSource.doLast {
    moveAntlrGeneratedFilesToTheirPackages(source, antlrGen, baseDir)
}

def moveAntlrGeneratedFilesToTheirPackages(FileTree grammarFiles, generatedFolder, destFolder) {
    grammarFiles.each {File file ->
        final grammarName = file.name.lastIndexOf('.') >= 0 ? file.name[0 .. file.name.lastIndexOf('.') - 1] : file.name
        final grammarPackage = extractPackageNameFromGrammerFile(file)
        copy {
            from generatedFolder
            include "${grammarName}*.*"
            into destFolder + "/" + grammarPackage.replaceAll("\\.", "/")
        }
    }
    project.delete fileTree(generatedFolder).include('*.*')
}

def extractPackageNameFromGrammerFile(File grammarFile) {
    def grammarPackage = "unknown.package"
    def packageRegex = ~/[ ]*package[ ]*([a-zA-Z]+[a-zA-Z0-9.-_]*)[ ]*;/
    grammarFile.eachLine { line ->
        def matcher = packageRegex.matcher(line)
        if (matcher.find()){
            grammarPackage = matcher.group(1)
        }
    }
    return grammarPackage
}

jar {
    baseName = 'XXX.YYY'
    version =  '1.0.0'
    manifest.attributes('Main-Class': mainClassName);
}

task wrapper(type: Wrapper) {
}

Test.java

package XXX.YYY;

import java.lang.Exception;
import org.antlr.v4.runtime.CharStreams;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import XXX.YYY.ZZZ.*;

public static void main(String[] args) throws Exception {

...

score 0 · Answer 4 · answered Jul 31 '15 at 15:29

I have similar issue using antlr3. You can fix the positions appending a post processing to the generateGrammarSource task:

/* Antlr generated files have wrong position so fix it with a post generation processing */
import groovy.io.FileType
generateGrammarSource << {
    logger.info("Antlr doesn't produce files to rigth place. Fix using package declaration")
    def move = [/*File wrong position*/:/*File rigth position*/]
    generateGrammarSource.outputDirectory.eachFileMatch( FileType.FILES, ~/.*\.java/ ) { f ->
        def pkg = f.readLines().find { it.trim() =~ /^package/ }
        if (pkg) {
            pkg = pkg.split( ' ' )[1].replace( ';','' )
            pkg = pkg.replace( '.','/' )
            move.put( f,new File("${generateGrammarSource.outputDirectory}/$pkg/${f.name}") )
        }
    }
    move.each { s,t ->
    logger.info( "Moving ${s.name} to right location .." )
    t.parentFile.mkdirs()
    s.renameTo( t.absolutePath )
}

score 0 · Answer 5 · edited Mar 19 '20 at 11:24

0

Put your grammar file in hu/pmi/wls/antlr/ws directory.

edited Mar 19 '20 at 11:24

colidyre

4,170
12
37
53

answered Mar 19 '20 at 10:17

MMR01

1

Generating ANTLR4 grammar files with package declaration in gradle

5 Answers5

antlr.g4

build.gradle

Test.java

Linked