Python Macros: Use Cases?

Question

If Python had a macro facility similar to Lisp/Scheme (something like MetaPython), how would you use it?

If you are a Lisp/Scheme programmer, what sorts of things do you use macros for (other than things that have a clear syntactic parallel in Python such as a while loop)?

Answer 1

This is a somewhat late answer, but MacroPy is a new project of mine to bring macros to Python. We have a pretty substantial list of demos, all of which are use cases which require macros to implement, for example providing an extremely concise way of declaring classes:

@case
class Point(x, y)

p = Point(1, 2)
print p.x   # 1
print p     # Point(1, 2)

MacroPy has been used to implement features such as:

• Case Classes, easy Algebraic Data Types from Scala
• Pattern Matching from the Functional Programming world
• Tail-call Optimization
• Quasiquotes, a quick way to manipulate fragments of a program
• String Interpolation, a common feature in many languages, and Pyxl.
• Tracing and Smart Asserts
• PINQ to SQLAlchemy, a clone of LINQ to SQL from C#
• Quick Lambdas from Scala and Groovy,
• Parser Combinators, inspired by Scala's.

Check out the linked page to find out more; I think I can confidently say that the use cases we demonstrate far surpass anything anyone's suggested so far on this thread =D

Answer 2

I believe that macros run counter to Python's culture. Macros in Lisp allow the big ball of mud approach; you get to redefine the language to become more suited to your problem domain. Conversely Pythonic code uses the most natural built in feature of Python to solve a problem, instead of solving it in a way that would be more natural in a different language.

Macros are inherently unpythonic.

Answer 3

Some examples of lisp macros:

• ITERATE which is a funny and extensible loop facility
• CL-YACC/FUCC that are parser generators that generate parsers at compile time
• CL-WHO which allows specifying html documents with static and dynamic parts
• Parenscript which is a javascript code generator
• Various simple code-wrappers, e.g., error handlers (I have a with-gtk-error-message-handler that executes code and shows GtkMessageDialog if unhandled error occurs), executors (e.g., given a code, execute it in different thread; I have a within-main-thread macro that executes code in different threads; PCall library uses macros to wrap code to be executed concurrently)
• GUI builders with macros (e.g., specify widgets hierarchy and widgets' properties and have a macro generate code for creation of all widgets)
• Code generators that use external resources during compilation time. E.g., a macro that processes C headers and generates FFI code or a macro that generates classes definitions based on database schema
• Declarative FFI. E.g., specifying the foreign structures, functions, their argument types and having macros to generate corresponding lisp structures, functions with type mapping and marshaling code
• Continuations-based web frameworks for Common Lisp use macros that transform the code into CPS (continuation passing style) form.

Answer 4

Here's one real-world example I came across that would be trivial with macros or real metaprogramming support, but has to be done with CPython bytecode manipulation due to absence of both in Python:

http://www.aminus.net/dejavu/chrome/common/doc/2.0a/html/intro.html#cpython

This is how the problem is solved in Common Lisp using a combination of regular macros, and read-macros to extend the syntax (it could have been done without the latter, but not the former):

http://clsql.b9.com/manual/csql-find.html

The same problem solved in Smalltalk using closures and metaprogramming (Smalltalk is one of the few single-dispatch OO languages that actually gets message passing right):

http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg02096.html

Here I tried to implement the Smalltalk approach in Common Lisp, which is a good illustration of how metaprogramming is poorly supported in the latter:

http://carcaddar.blogspot.com/2009/04/closure-oriented-metaprogramming-via.html

Answer 5

There's a mailing list posting (archive.org mirror) which explains this rather well. The post is about Perl, but it applies to Python just as well.

Answer 6

In lisp, macros are just another way to abstract ideas.

This is an example from an incomplete ray-tracer written in clojure:

(defmacro per-pixel
  "Macro.
Excecutes body for every pixel. Binds i and j to the current pixel coord."
  [i j & body]
  `(dotimes [~i @width]
     (dotimes [~j @height]
       ~@body)))

If you want to do something to every pixel with coordinates (i,j), say, draw a black pixel if i is even, you would write:

(per-pixel i,j
  (if (even? i)
    (draw-black i,j)))

This is not possible to do without macros because @body can mean anything inside (per-pixel i j @body)

Something like this would be possible in python as well. You need to use decorators. You can't do everything you can do with lisp macros, but they are very powerful

Check out this decorator tutorial: http://www.artima.com/weblogs/viewpost.jsp?thread=240808

Answer 7

Some uses cases I have seen before include making class factories or stripping logging statements out of production code.

Answer 8

I'd use it to wrap yield to enable me to build more powerful generator pipelines.

Answer 9

Currently, the only way features can be added to Python the language is through a PEP (Python Enhancement Proposal). This can be slow, and doesn't help you in the cases when you want to add a feature to the language that is only useful for your use case.

For example, there's a PEP to add a do-while loop. This will probably be added to Python, but the PEP was created in 2003. I'd like to write do-while loops today, and I could do that if Python had macros.

Similarly, there was a PEP to add labelled break and continue but this was rejected. If labelled break statements would make my code clearer, I could implement them with a macro.

PEPs aside, I would also like an unless macro. Rather than writing:

if not is_superman():
    dodge_bullet()

I could write:

unless is_superman():
    dodge_bullet()

I'd like a case macro (often called cond in Lisp). Rather than writing:

if x == FOO:
    do_stuff_with_foos()
elif x == BAR:
    do_stuff_with_bars()
elif x == BAZ:
    do_stuff_with_bazs()

I could write:

switch x:
   case FOO:
       do_stuff_with_foos()
   case BAR:
       do_stuff_with_bars()
   case BAZ:
       do_stuff_with_bazs()

These would be straightforward to implement as macros. More complex, useful macros would include:

• Ruby style string interpolation e.g. "hello there {user}" (probably best implemented as a reader macro)
• Pattern matching

Currently, these are only features in other languages. With macros, I could add them to Python. I could even write PEPs that included an example implementation. (Some PEPs already do this, but they are forced to modify the C source of the interpreter itself.)

Answer 10

I don't think Python needs macros, because they are useful for 2 things:

1. Creating a DSL or more eloquent syntax for something (Lisp LOOP macro is a nice example). In this case, Python philosophy decided against it deliberately. If there is some explicit notation you're missing, you can always ask for a PEP.

2. Making things faster by precomputing things at compile time. Python isn't oriented to speed, so you can always use a function instead.

I am not saying macros are wrong, just that they don't fit Python philosophy. You can always do without them without much code duplication, because you have duck typing and operator overloading.

And as a side note, I would much rather see Lisp's restarts in Python than macros.

Answer 11

I want to use macro to enable sql statement in python code. - select * from table1

Answer 12

Hy, For my own use, I created a Python module (Espy) that allows macro definitions with arguments, loop and conditional code generation: You create a source.espy file, then launch the appropriate function, then source.py is generated.

It allows syntaxes as following:

macro repeat(arg1):
    for i in range(%arg1%):
        socket
    print "stop"
   ...
repeat(5):
    print "Hi everybody"
    print "See you soon"

is equivalent to:

...
for i in range(5):
    print "Hi everybody"
    print "See you soon"
print "stop"

Other syntax:

macro doit(arg1):
    for i in %arg1%:
        socket suit(arg2):
            socket
            print %arg2%
        socket check(arg3):
            if %arg2%==%arg3%:
                socket
...
#use
doit(range(10)):
    suit(result):
        result=i*i
    check(result,25):
        print "I knew that 5*5 == 25"

is equivalent to:

for i in range(10):
    result=i*i
    print result
    if result==25:
        print "I knew that 5*5 == 25"

More, Espy has 2 functions: "macro for" and "macro if". An example:

macro for v in [6,10,12,20,23]:
    macro if 7<%v%<22:
        True:
            print "At %v%, I'm awake."
        False:
            print "At %v%, I'm sleeping."

is translated by Espy in:

print "At 6, I'm sleeping."
print "At 10, I'm awake."
print "At 12, I'm awake."
print "At 20, I'm awake."
print "At 23, I'm sleeping."

Complete documentation and free download can be found here: http://elp.chronocv.fr

I use this module in many cases. It permits more structured and shorter codes. With it I generated 65000 lines of clear and efficient python code from 1000 lines of espy code for a new chess engine project (still in progress).

If Python could include macros in futur release, it'd become more impressive.

Answer 13

Well, I'd like instead of

print >> sys.stderr, "abc"

to write

err "abc"

in some scripts which have many debug printout statements.

I can do

import sys
err = sys.stderr

and then

print >> err, "abc"

which is shorter, but that still takes too many characters on the line.

Answer 14

Coming from a C-world, I'd really appreciate being able to do efficient logging of rich messages. Instead of writing

if logger.level > logger.DEBUG:
    logger.log('%s' % (an_expensive_call_that_gives_useful_info(),))

with macros, one could instead do something like

DEBUG('%s' % (an_expensive_call_that_gives_useful_info(),))

Answer 15

Possibly if you want the source code at runtime such as for debugging (say printf debugging an expression's value with the name of it so you don't have to write it twice).

The only way I could think of to do it in python is to pass a string to eval.

Answer 16

Read "The Lambda Papers" so you might find out generally why one would take advtage of macros at all.

You should start with ‘AIM-353 Lambda:The Ultimate Imperative’ and follow it with ‘AIM-443 Lambda: The Ultimate GOTO’. Both may be found here:

http://library.readscheme.org/page1.html