3

I have been programming in python exclusively for 4 years and have never really looked under the hood at the C code in which python is written. I have recently been looking into a problem that would involve modifying python at that level.

The code seems pretty consistent, and thus relatively easily understood. However, it's complex enough that it wasn't making sense to me just by studying it how it all worked together. Granted, I didn't spend a lot of time or effort on that, for want of a better resource. I also looked over the documentation on the python site. However, it is oriented more toward extending the language through modules.

I was hoping to find some straightforward documentation on how the parser works at the C level and how to extend the core language directly (adding language features). The module-oriented documentation provides some great insight into the way types are built and objects are managed, but I am looking for more.

Is there any such documentation out there?

Community
  • 1
  • 1
Eric Snow
  • 1,198
  • 8
  • 21
  • 2
    read the source, Luke - the ultimate documentation – pm100 Aug 17 '10 at 17:13
  • 4
    Eli Bendersky has written this fantastic tutorial on doing exactly what you're asking about: [Python internals: adding a new statement to Python](http://eli.thegreenplace.net/2010/06/30/python-internals-adding-a-new-statement-to-python/). I'd post this as an answer, but since Eli is a very prolific member of the SO community, surely he should be the one to do it, should he so choose. – Michał Marczyk Aug 17 '10 at 17:27
  • @pm100, yeah, I started to and then decided I better check for more documentation before I spent too much time on it unnecessarily. That said, I know there are operational details you learn from the code that I have not found documented elsewhere. – Eric Snow Aug 17 '10 at 17:33
  • @Michal (and Eli), that is a great resource. I am going to start there. Thanks! – Eric Snow Aug 17 '10 at 17:36
  • 1
    @Michał: cheers. I've actually posted my answer *before* I saw this comment ;-) – Eli Bendersky Aug 17 '10 at 18:03

3 Answers3

3

This article may help you get started. It takes a lot of information from the excellent PEP 339 - Design of the CPython Compiler.

Eli Bendersky
  • 263,248
  • 89
  • 350
  • 412
1

http://docs.python.org/extending/index.html - Custom modules/extensions

http://docs.python.org/c-api/index.html - C API, under the hood

Ian Wetherbee
  • 6,009
  • 1
  • 29
  • 30
1

There's not too much written lore on this topic. Your best bet is to just simply follow the guidelines in PEP 306

SingleNegationElimination
  • 151,563
  • 33
  • 264
  • 304