Intellisense from PEG (Parsing expression grammar)

Question

I apologise in advance if this has already been asked. I have a language, defined by a grammar, and I'd to know how people go about implementing Intellisense for their custom grammars. This seems mechanical to me; the user types something that is then fed to the generated parser and it offers keyword suggestions. I guess parsing will need to be modified so that it is incremental instead of being one-off, i.e., the generated parser.

I'm new to this area so any tips are welcome.

I'm intending to use http://pegjs.majda.cz/ but anything will do.

First thing to mention is that PEG.js is in no way incremental parser. Second, I've written a library inspired by PEG.js, and I'm currently saturating it with features before publishing. Incremental parsing is a hard moment, so I went googling for some more info. Last, look into CodeMirror's samples. The thing you're looking for is implemented absolutely in a straightforward way: on every edit parse the code, save all identifiers of a certain kind into data structures (arrays if you don't have objects in the language), look up. — polkovnikov.ph, Sep 04 '14 at 12:10
possible duplicate of [How to write a Visual Studio extension for a template or markup language that supports embedded code snippets](http://stackoverflow.com/questions/5849205/how-to-write-a-visual-studio-extension-for-a-template-or-markup-language-that-su) — Paul Sweatte, Oct 13 '14 at 22:06

score 1 · Answer 1 · answered Oct 16 '14 at 03:42

There is a codemirror grammar add-on i have written for some projects which required syntax-highlight for various custom languages.

One defines a grammar in JSON format for any language (or variation since json grammars can extend other json grammars) and the add-on creates automaticaly a codemirror parser which can include syntax parts and syntax-errors, autocompletion and more

See live example here

NOTE: The json grammar format used in the add-on is similar in many ways to a PEG grammar formalism (maybe with more options). but used mainly for generating syntax-highlight parsers. The project is evolving and more options will probably be added in the future.

score 0 · Answer 2 · answered Oct 07 '22 at 17:05

I think it will depend on what your parser returns. For example the pegged library in the D programming language. It's usage looks like:

language.d:

module language;

import pegged.grammar;

enum Language = `
Math:
   Content < Sentence+
   Sentence < LetStmt "."
   LetStmt  < Let Var Be A Vocab
   Vocab < FormWord / Word
   FormWord <- HyphenForm "-" Word / Word "-" HyphenForm
   HyphenForm < ("(" Form ")" / Var / ZZ)
   Word <- LatinWord
   LatinWord <~ LatinAlpha+ 
   Var <- identifier
   LatinAlpha <- [a-zA-Z]
   ZZ <~ [0-9]+
   Form <~ .+
   Let <- "Let" / "let" 
   Be <- "be"
   A <- "An" / "an" / "A" / "a"   
`;

mixin(grammar(Language));

app.d:

import std.stdio;

import language;

void main()
{
    while (true) {
        auto text = readln();
        auto parseTree = Math(text);
        writeln(parseTree);   
   }

}

Console I/O:

Let G be a group
Math (failure)
 +-Math.Content (failure)
    +-Math.Sentence (failure)
       +-Math.LetStmt[0, 17]["Let", "G", "be", "a", "group"]
       |  +-Math.Let[0, 4]["Let"]
       |  +-Math.Var[4, 6]["G"]
       |  +-Math.Be[6, 9]["be"]
       |  +-Math.A[9, 11]["a"]
       |  +-Math.Vocab[11, 17]["group"]
       |     +-Math.Word[11, 17]["group"]
       |        +-Math.LatinWord[11, 16]["group"]
       +-literal!(".") Failure at line 1, col 0, after "e a group\n" expected "\".\"", but got ""


Let G be a group.
Math[0, 18]["Let", "G", "be", "a", "group", "."]
 +-Math.Content[0, 18]["Let", "G", "be", "a", "group", "."]
    +-Math.Sentence[0, 18]["Let", "G", "be", "a", "group", "."]
       +-Math.LetStmt[0, 16]["Let", "G", "be", "a", "group"]
          +-Math.Let[0, 4]["Let"]
          +-Math.Var[4, 6]["G"]
          +-Math.Be[6, 9]["be"]
          +-Math.A[9, 11]["a"]
          +-Math.Vocab[11, 16]["group"]
             +-Math.Word[11, 16]["group"]
                +-Math.LatinWord[11, 16]["group"]

As you can see, the first time I entered "Let G be a group", I forgot the period at the end of the sentence. This resulted in a parse failure. At least printing the parseTree will show this, perhaps traversing the parseTree will also let you see the failure.

When there is a failure it shows "what's expected". Isn't this what's expected the part you would show in an intellisense drop-down?

I'm not sure myself. This post is just a thought about the problem.

Another and I think better option is to use the PEG parser example provided by the pegged developer. You can then take your grammar, parse it into a parseTree and that gives you the "tree" required for one technique of doing intellisense. — MathCrackExchange, Oct 07 '22 at 17:19

Intellisense from PEG (Parsing expression grammar)

2 Answers2

language.d:

app.d:

Console I/O: