How do you match zero or more tokens in Jison?

Question

I'm writing a simple expression parser in Jison allowing an arbitrary number of newlines to follow a binary operator in an expression. This is my grammar so far:

{
    "operators": [
        ["left", "+", "-"],
        ["left", "*", "/", "%"]
    ],
    "bnf": {
        "program": [
            ["statement EOF", "return $1;"]
        ],
        "statement": [
            ["expression newlines", "$$ = $1 + ';';"]
        ],
        "expression": [
            ["NUMBER",                           "$$ = yytext;"],
            ["expression + expression",          "$$ = $1 + ' + ' + $3;"],
            ["expression - expression",          "$$ = $1 + ' - ' + $3;"],
            ["expression * expression",          "$$ = $1 + ' * ' + $3;"],
            ["expression / expression",          "$$ = $1 + ' / ' + $3;"],
            ["expression % expression",          "$$ = $1 + ' % ' + $3;"],
            ["expression + newlines expression", "$$ = $1 + ' + ' + $4;"],
            ["expression - newlines expression", "$$ = $1 + ' - ' + $4;"],
            ["expression * newlines expression", "$$ = $1 + ' * ' + $4;"],
            ["expression / newlines expression", "$$ = $1 + ' / ' + $4;"],
            ["expression % newlines expression", "$$ = $1 + ' % ' + $4;"]
        ],
        "newlines": [
            ["NEWLINE",          ""],
            ["newlines NEWLINE", ""]
        ]
    }
}

As you can see I'm writing two rules for every binary operator. That seems to me to be very redundant. I would rather have a production which matches zero or more NEWLINE tokens (Kleene star) instead of one or more tokens (Kleene plus). How would you do this in Jison?

Can you make the `newlines` production include a null terminal? — Barmar, Apr 04 '13 at 04:22
@Barmar - I can, but I don't really see why I should. In all probability you'll never encounter a null character in a regular text file. Plus we are not dealing with C strings here. — Aadit M Shah, Apr 04 '13 at 04:58
I didn't mean a null character, I meant an empty production. — Barmar, Apr 04 '13 at 05:00
@Barmar - Unfortunately, no. It seems Jison doesn't support empty productions. I'll need to do some more digging. — Aadit M Shah, Apr 04 '13 at 05:03
@Barmar - Oops, I seem to have been going about it the wrong way. Jison _does_ support empty productions. Instead of changing the production `newlines -> NEWLINE | newlines NEWLINE` to `newlines -> | newlines NEWLINE`, I added a new production to form `newlines -> | NEWLINE | newlines NEWLINE`. This was what was causing the problem. Would you write your comment in the form of an answer so that I may accept it? — Aadit M Shah, Apr 04 '13 at 05:40

score 1 · Accepted Answer · answered Jul 01 '13 at 15:32

I use Jison and I ignore white-space (including new-lines).

The first line in my %lex is:

\s+   /* ignore */

But you don't have to do it that way if you don't want to. Try something along these lines:

"expression": [
            ["NUMBER",                           "$$ = yytext;"],
            ["expression + expression",          "$$ = $1 + ' + ' + $3;"],
            ["expression - expression",          "$$ = $1 + ' - ' + $3;"],
            ["expression * expression",          "$$ = $1 + ' * ' + $3;"],
            ["expression / expression",          "$$ = $1 + ' / ' + $3;"],
            ["expression % expression",          "$$ = $1 + ' % ' + $3;"],
            ["expression newlines",              "$$ = $1"],
            ["newlines expression",              "$$ = $2"]
        ],

That should allow any amount of new lines before/after any expression.

How do you match zero or more tokens in Jison?

1 Answers1