1

So as stated in the title, my task is to traverse the Parse Tree generated for code written in Java (grammar is a standard Java grammar), print most of it unchanged and modify only some words, for example type declarations.

My current approach was to create ParseTreeListener and implement the logic in the enterEveryRule method, but unfortunately it doesn't appear to work even for basic printing. The output is very messy and there are a lot of repetitions, as if every node was visited multiple times.

My another try was to implement appropriate methods in BaseListener that would do the changes to the type declarations I need, but from there I see no possibility to print the rest of the code unchanged.

Looking forward to your help!

Broccoli
  • 1,249
  • 1
  • 13
  • 19
  • Have you tried using the walker to visit the parse tree? – Mike Lischke Jun 16 '16 at 06:39
  • First give some code, grammar, what you've done etc.. It's hard to say what is not working. It's quite a basic thing - walking the tree with ANTLR4. Secondly, you don't need to create `ParseTreeListener`, nor implement `BaseListener` but override methods in generated listener. Or as suggested by @MikeLischke use visitor - again you'd need to override generated visitor.. – cantSleepNow Jun 17 '16 at 09:07
  • 1
    Show us the code you used for printing the messy output. Mostly likely you have incorrectly used RuleContext.getText() which is no good for printing out the raw input text since normally your grammar instructed the lexer to remove all whitespace/CRLF. Did your listener class use BufferedTokenStream for printing the original input? – JavaMan Jun 18 '16 at 07:12
  • 1
    Hey guys, thanks for your answers, I had to leave my work for a few days because of a wedding that took place in another city but here's what I managed to figure out before leaving: 1. The mess, and what I believe was visiting one node multiple times were caused by using `enterEveryRule` method not `visitTerminal` method and using `getText()` there. 2. My another problem was exactly what @JavaMan suggested which was having all the whitespaces skipped by the lexer. My last attempt was to modify the grammar but it appeared too complicated to do. I am gonna get back to it in a few hours – Broccoli Jun 21 '16 at 10:50
  • @JavaMan - regarding your question, I used CommonTokenStream. Any ideas? :) – Broccoli Jun 21 '16 at 11:00

1 Answers1

0

You could use ANTLR's string templates to produce code from the ASTs.

In general, you start with set of "standard" string templates that can regenerate source code corresponding to the underlying tree.

To get the effect you want, you judiciously choose the standard string templates on AST nodes where you don't want changes, and variant templates where you do want changes.

IMHO, it is better to modify the AST, and then simply apply the standard templates.

Ira Baxter
  • 93,541
  • 22
  • 172
  • 341
  • 1
    Well, there is no AST produced by ANTLR4. – Mike Lischke Jun 16 '16 at 06:38
  • I use tools other than ANTLR. I use the term loosely; ok, parse tree for ANTLR. The tools I use minimize this distinction; see http://stackoverflow.com/a/1916687/120163. Thanks for the reminder about how ANTLR specifically works. . Doesn't change the essentials of the answer. – Ira Baxter Jun 16 '16 at 07:09
  • 1
    Not questioning your competency in general, but this answer is wrong. There is not only no AST, but you cannot modify the parse tree either (which is a completely different thing than an AST). In ANTLR you would probably use a listener or visitor that do the job (with or without string templates), instead of generating an AST first that you then transform. – Mike Lischke Jun 16 '16 at 09:32
  • ANTLR seems to have funny rules. If you want to manipulate a tree (parse, AST, or other) and your manipulations are anything beyond trivial, it is easier in practice to modify that tree and then "print" the modified result, especially if the modifications compound. We can argue about what to do if the changes are trivial, but this is really an argument about what is "trivial" not general practice. Parse tree not modifiable in ANTLR? Well you can copy it inexactly (to produce an AST) or exactly (to produce an AST isomorphic to the parse tree) and then make your changes. – Ira Baxter Jun 16 '16 at 14:23
  • I've been not precise with my previous comment, but I cannot change it anymore. Of course you could manipulate a parse tree in ANTLR, but that won't help for the task at hand, because a parse tree represents the path the parser has taken through its rules, while a syntax tree represents the syntactic structure of the input (at least in ANTLR). Of course, you can create your own AST and work with that, no doubt. It's just that ANTLR doesn't do it automatically. – Mike Lischke Jun 16 '16 at 15:55
  • OP is going to have to decide if he wants to implement something based on "trivial" changes which he didn't make to (some) tree [the mapping that tracks this will be a mess, probably] or whether he wants to go the tree modification route. [no ugly mapping to maintain]. The fact that ANTLR does not support modifiable-tree building automatically is a feature of the technology choice he has made; he either slugs it out or switches technologies to something else, where he may or may not have other issues. – Ira Baxter Jun 16 '16 at 19:13