9

I'd like to visualize a grammar file (actually the Jison grammar for coffee-script). So the input file is a grammar file of Bison/Yacc style. The expected output could be a Graphviz dot file or something similar.

I'm not necessarily looking for a complete IDE, like GOLD. But it's important to be able to handle a LALR input, that's why the excellent ANLTRWorks doesn't come into account.

I also checked a comparison of parsers on Wikipedia, but it includes only IDE support, but not visualization.

This is the coffeescript grammar file I actually want to visualize.

Kevin Brown-Silva
  • 40,873
  • 40
  • 203
  • 237
Adam Schmideg
  • 10,590
  • 10
  • 53
  • 83
  • 2
    Are you asking for visualization as a [syntax diagram](http://en.wikipedia.org/wiki/Syntax_Diagram)? – Gunther Nov 16 '11 at 18:10
  • @Gunther, yes, it's one sort of visualization I can imagine. Thanks for the link. The referred page contains some links to tools I'm going to check out. – Adam Schmideg Nov 17 '11 at 14:22
  • 2
    Well, there is [the one that I wrote](http://railroad.my28msec.com/). It accepts W3C style grammar syntax and is not tied to a particular parsing strategy. Converters from other grammar notations (e.g. ABNF, ANTLR, Bison, GOLD, JavaCC, Jison, PEG.js, yacc) are under construction, though. If you have a little time, I could make the one for Jison available within a few days. Could not find a .jison file of the CoffeeScript grammar, though. Is the grammar only assembled at runtime? – Gunther Nov 17 '11 at 15:22
  • @Gunther, that sounds really generous. I updated my question with a link to the grammar file. – Adam Schmideg Nov 17 '11 at 17:44
  • Please note that the Railroad Diagram Generator has moved to http://bottlecaps.de/rr/ui – Gunther Dec 05 '13 at 12:46

2 Answers2

13

Here are the instructions for creating a syntax diagram.

The content of grammar.coffee is executable code, which must be run for getting the actual Jison grammar. I used the Try CoffeeScript page to compile it, after having replaced the Jison call by a Javascript alert. Then ran the resulting Javascript to obtain the grammar, which looks like this:

{
  "tokens":" TERMINATOR TERMINATOR TERMINATOR STATEMENT INDENT OUTDENT INDENT OUTDENT IDENTIFIER NUMBER STRING JS REGEX BOOL = = INDENT OUTDENT : : INDENT OUTDENT RETURN RETURN HERECOMMENT PARAM_START PARAM_END -> =>  ,  , ... = ... . ?. :: :: INDEX_START INDEX_END INDEX_SOAK { }  , TERMINATOR INDENT OUTDENT CLASS CLASS CLASS EXTENDS CLASS EXTENDS CLASS CLASS CLASS EXTENDS CLASS EXTENDS SUPER SUPER  FUNC_EXIST CALL_START CALL_END CALL_START CALL_END THIS @ @ [ ] [ ] .. ... [ ] , TERMINATOR INDENT OUTDENT INDENT OUTDENT , TRY TRY TRY FINALLY TRY FINALLY CATCH THROW ( ) ( INDENT OUTDENT ) WHILE WHILE WHEN UNTIL UNTIL WHEN LOOP LOOP FOR FOR FOR OWN , FORIN FOROF FORIN WHEN FOROF WHEN FORIN BY FORIN WHEN BY FORIN BY WHEN SWITCH INDENT OUTDENT SWITCH INDENT ELSE OUTDENT SWITCH INDENT OUTDENT SWITCH INDENT ELSE OUTDENT LEADING_WHEN LEADING_WHEN TERMINATOR IF ELSE IF ELSE POST_IF POST_IF UNARY - + -- ++ -- ++ ? + - MATH SHIFT COMPARE LOGIC RELATION COMPOUND_ASSIGN COMPOUND_ASSIGN INDENT OUTDENT EXTENDS",
  "bnf":
  {
    "Root":
    [
      ["","return $$ = new yy.Block;",null],
      ["Body","return $$ = $1;",null],
      ["Block TERMINATOR","return $$ = $1;",null]
    ],
    "Body":
    [
      ["Line","$$ = yy.Block.wrap([$1]);",null],
      ["Body TERMINATOR Line","$$ = $1.push($3);",null],
      ["Body TERMINATOR","$$ = $1;",null]
    ],
    "Line":
    [
      ["Expression","$$ = $1;",null],
      ["Statement","$$ = $1;",null]
    ],
    ...

The above can be fed to the Jison-to-W3C grammar converter, resulting in a grammar like this:

Root     ::= ( Body | Block TERMINATOR )?
Body     ::= Line ( TERMINATOR Line | TERMINATOR )*
Line     ::= Expression
           | Statement
...

From here we can have the Railroad Diagram Generator create a syntax diagram:

CoffeeScript Syntax Diagram

. . .

Note that the converter only evaluates the "bnf" part of the grammar, so it does not take the token definitions into account. This could be improved by doing some manual postprocessing of the W3C-style grammar.

Gunther
  • 5,146
  • 1
  • 24
  • 35
  • 1
    It's just wonderful, thank you very much. Just accepting it and voting it up seems so little :) I used `JSON.stringify(grammar)` instead of an alert. – Adam Schmideg Nov 22 '11 at 21:33
  • @AdamSchmideg, glad to know that you like it. Keep in mind that the conversion tool is still under development, and far from perfect. However I will make serious attempts to repair any reported deficiencies. – Gunther Nov 26 '11 at 18:28
  • care to post, like, the complete source you fed into the grammar converter? such a RR diagram would be utterly useful for me! i managed to make grammar.js spill its representation, i did wrap it like you show, but still the converter complains that "lexical analysis failed while expecting IdentifierName at line 413, column 1" which happens to be EOF. or paste the entire grammar for simplicity? i'd love to do it myself, tho. – flow Jan 19 '12 at 17:08
  • As it looks for IdentifierName, it failed on jison input and was trying to parse a PEG.js grammar. So presumably something was wrong with the JSON syntax. Here is how to do it using jrunscript and json2.js. From grammar.coffee, remove the `{Parser} = require 'jison'` and replace the parser call by `load("json2.js");println(JSON.stringify({tokens: tokens.join(' '), bnf: grammar, operators: operators.reverse(), startSymbol: 'Root'}));`. Then compile it on `Try Coffescript`, and run the resulting js with jrunscript. This will create JSON as suitable for the converter. Will email the artifacts. – Gunther Jan 19 '12 at 19:58
  • looking forward to your mail. isn't this setup a tad overcomplicated? i mean what's json2 anyway? i can `npm install` it but it wouldn't even load but fails with `Class is not defined`—fixed that one by `require`ing `joose` first. it still wouldn't run in nodejs (not sure why i should use jrunscript—that fails, too, coz it can't load json2). it must be possible to get this right in a much more straightforward way. – flow Jan 20 '12 at 17:22
  • Well, you asked for support on getting this done, and that's what came into my mind, plus I have proved that it works this way. Can't see why you think it is too complicated. jrunscript was proposed because it is a ubiquitous tool for running javascript, part of the JDK after all. Feel free to use other platforms as you like. As for json2.js, you just need a single file from [JSON-js](https://github.com/douglascrockford/JSON-js), and this is to satisfy the reference to `JSON.stringify`. You should have received my Thursday email. Let's discuss further details offline. – Gunther Jan 21 '12 at 11:49
0

so i tried again and found my most blatant mistake right away—the json i had posted was incorrectly using single instead of double quotes. let me detail the workflow; it's simple enough, and if you're already running CoffeeScript on NodeJS you're ready to go:

  • locate the node_modules/coffee-script/lib/coffee-script/grammar.js module in your file system;

  • copy & paste the code of that file into the source pane of the js->coffee pane on the js2coffee site (you could skip that, but i find it much more agreeable to edit CS than to fiddle with JS).

  • save the translated code to node_modules/coffee-script/lib/coffee-script/grammar.coffee;

  • go and locate

    exports.parser = new Parser(
      tokens: tokens.join(" ")
      bnf: grammar
      operators: operators.reverse()
      startSymbol: "Root"
    )
    

    in the code; replace it with

    console.log JSON.stringify
      tokens: tokens.join " "
      bnf: grammar
      operators: operators.reverse()
      startSymbol: "Root"
    

    while taking care to use the exact same indentation (two space for the first line, four for the rest).

  • from the command line, run sth like coffee node_modules/coffee-script/lib/coffee-script/grammar.coffee > /tmp/coffee.grammar;

  • copy and paste the code of the resulting file into the grammar converter;

  • copy and paste the resulting EBNF grammar from the converter into the grammar editor over at the railroad diagram generator;

  • go over to the View Diagram tab and — rejoice!

it's sort of a chore to do all of this copy'n'pastish stuff, but certainly good enough for any one-off visualization. i've been searching the web a lot for a reasonable RR diagram generator, and this particular one is definitely among the ones with the prettiest output. sort of surprising when you think of how simple railroad diagrams really are.

Gunther
  • 5,146
  • 1
  • 24
  • 35
flow
  • 3,624
  • 36
  • 48