7

I'm using Sprache monadic parser to parse a DSL.

This is a snippet of my grammar:

public static readonly Parser<IExpression> TerminatedStatement =
    from exp in Parse.Ref(() => Expression)
    from _ in Parse.Char(';').Token()
    select exp;

public static readonly Parser<IExpression> Statement =
    Parse.Ref(() => TerminatedStatement)
    .Or(Parse.Ref(() => InvocationStatement));

public static readonly Parser<Statements> Statements =
    from statements in Statement.Many()
    select new Statements(statements);

if I then use that with Statements.Parse(" ") I get an exception saying that there was an unexpected end of input.

How can that be when the Statements use the Many operator, which AFAIK yields 0-n results.

" " should return a Statements instance containing 0 statements.

So how can the parser complain that there is an unexpected end of input? shouldnt it just conclude that there are no statements there? (no matter what funky stuff the different expressions that make up the statements do)

CSDev
  • 3,177
  • 6
  • 19
  • 37
Roger Johansson
  • 22,764
  • 18
  • 97
  • 193
  • Normally in a PG you have to indicate where the file end is expected, else it will just parse till it is happy. This may or may not be desirable. Try adding another `Or` in `Statement` with `End` (that seems from the code what is needed). – leppie Jul 13 '15 at 12:06
  • I've tried to reproduce this with a few variations like `Parse.String("aa").Many().Parse("abbb")` - all of which succeed with `[]` as you expect... Any chance of a repro/test case in https://github.com/sprache/sprache perhaps? Cheers! – Nicholas Blumhardt Jul 13 '15 at 22:43
  • 1
    @NicholasBlumhardt, I'm also seeing this bug. It seems to have to do with having a statement that has multiple initial tokens. So if you have IfStatement.Or(WhileStatement).Many(), it works fine, because each option has a unique selector ('if' or 'while'). But add .Or(Expression), whcih can start with an identifer or a number or a string or a '(' and it fails with the unexpected EOI showing all the start tokens. The test case I use is the empty string. I'd post test, but the grammar is huge and I havn't got a small test case that fails yet. – Jerry Jul 16 '15 at 15:05
  • @Jerry Yepp, that would describe what Im seeing. The exact case i had was Identifier vs Identifier + "(" args ")" eg. a variable vs a method call. – Roger Johansson Jul 16 '15 at 16:18
  • Sounds like a bug lurking! I'd love to dig in and try to repro too, but very limited time in the next few weeks; if you have any luck with a minimal test I'd hope a fix could be turned around pretty quickly on our end. – Nicholas Blumhardt Jul 16 '15 at 22:07
  • In the Sprache. code, file Parse.cs line 423/424, there is a test if (fr.Remainder.Equals(i)). If the first parser matches the empty string it still invokes the second parser. Deleting these two lines appears to fix the problem. @NicholasBlumhardt, what is the reasoning for testing the second parser if the first succeeded with a zero-length match? – Jerry Jul 17 '15 at 14:34
  • Well spotted; I've raised a ticket here: https://github.com/sprache/Sprache/issues/52 including a possible workaround. Any help implementing and verifying a fix would be great. Cheers! – Nicholas Blumhardt Jul 19 '15 at 21:54

0 Answers0