51

I'm after creating a simple mini-language parser in Python, programming close to the problem domain and all that.

Anyway, I was wondering how the people on here would go around doing that - what are the preferred ways of doing this kind of thing in Python?

I'm not going to give specific details of what I'm after because at the moment I'm just investigating how easy this whole field is in Python.

Asclepius
  • 57,944
  • 17
  • 167
  • 143
Reality
  • 613
  • 1
  • 8
  • 8
  • 2
    Similar to: http://stackoverflow.com/questions/2945357/python-how-best-to-parse-a-simple-grammar – 0 _ Jul 24 '13 at 16:58

6 Answers6

43

Pyparsing is handy for writing "little languages". I gave a presentation at PyCon'06 on writing a simple adventure game engine, in which the language being parsed and interpreted was the game command set ("inventory", "take sword", "drop book", etc.). (Source code here.)

You can also find links to other pyparsing articles at the pyparsing wiki.

PaulMcG
  • 62,419
  • 16
  • 94
  • 130
  • 6
    Pyparsing is an absolutely fantastic library, you can parse next to anything with minimal ceremony. I built a recursive PHP parser (static code analysis) in two afternoons using it. I couldn't recommend it strongly enough. The docs take about 2 hours to read. – Bryan Hunt Dec 21 '12 at 16:17
26

I have limited but positive experience with PLY (Python Lex-Yacc). It combines Lex and Yacc functionality in a single Python class. You may want to check it out.

Fellow Stackoverflow'er Ned Batchelder has a nice overview of available tools on his website. There's also an overview on the Python website itself.

Community
  • 1
  • 1
Stephan202
  • 59,965
  • 13
  • 127
  • 133
22

I would recommend funcparserlib. It was written especially for parsing little languages and DSLs and it is faster and smaller than pyparsing (see stats on its homepage). Minimalists and functional programmers should like funcparserlib.

Edit: By the way, I'm the author of this library, so my opinion may be biased.

Ib33X
  • 6,764
  • 4
  • 28
  • 30
Andrey Vlasovskikh
  • 16,489
  • 7
  • 44
  • 62
  • 6
    At one time I was reluctant to promote or advocate for my pyparsing module - don't be bashful! And be sure that you write to Ned Batchelder to have `funcparserlib` added to his parser compendium page, it is a common (and valuable) resource for many Python users in search of a parsing library. – PaulMcG Oct 19 '09 at 06:45
  • Seems a very useful library, and the code looks very clear! Thanks! – culebrón Jan 03 '10 at 10:51
6

Python is such a wonderfully simple and extensible language that I'd suggest merely creating a comprehensive python module, and coding against that.

I see that while I typed up the above, PLY has already been mentioned.

gnud
  • 77,584
  • 5
  • 64
  • 78
4

If you ask me this now, I would try the textx library for python. You can very easily create a dsl in that with python! Advantages are that it creates an AST for you, and lexing and parsing is combined.

http://igordejanovic.net/textX/

Windel
  • 499
  • 4
  • 11
3

In order to be productive, I'd always use a parser generator like CocoPy (Tutorial) to have your grammar transformed into a (correct) parser (unless you want to implement the parser manually for the sake of learning).

The rest is writing the actual interpreter/compiler (Create stack-based byte code or memory AST to be interpreted and then evaluate it).

Dario
  • 48,658
  • 8
  • 97
  • 130