1

I keep reading that Lisp macros are one of the most powerful features of the language. But reading over the specifications and manuals, they are just functions whose arguments are unevaluated.

Given any macro (defmacro example (arg1 ... argN) (body-forms)) I could just write (defun example (arg1 ... argN) ... (body-forms)) with the last body-form turned into a list and then call it like (eval (example 'arg1 ... 'argN)) to emulate the same behavior of the macro. If this were the case, then macros would just be syntactic sugar, but I doubt that syntactic sugar would be called a powerful language feature. What am I missing? Are there cases where I cannot carry out this procedure to emulate a macro?

Isabella
  • 167
  • 3
  • There are already good answers below, but: "while/for loops are just syntactic sugar for GOTO". Even more: "programming languages are just syntactic sugar for assembly". Or "assembly is syntactic sugar for binary alphabet, single-taped, one-sided Turing machine", "mathematical notations are just syntactic sugar for [...]" etc etc. This is obviously sarcasm, but /even if/ macros were "just" syntactic sugar, this would say *nothing* about their "power"/usefulness. – Numbra Nov 29 '22 at 17:56

2 Answers2

3

I can't talk about powerful because it can be a little bit subjective, but macros are regular Lisp functions that work on Lisp data, so they are as expressive as other functions. This isn't the case with templates or generic functions in other languages that rely more on static types and are more restricted (on purpose).

In some way, yes macros are simple syntactic facilities, but you are focused in your emulation on the dynamic semantics of macros, ie. how you can run code that evaluates macros at runtime. However:

  • the code using eval is not equivalent to expanded code
  • the preprocessing/compile-time aspect of macros is not emulated

Lexical scope

Function, like +, do not inherit the lexical scope:

(let ((x 30))
  (+ 3 4))

Inside the definition of +, you cannot access x. Being able to do so is what "dynamic scope" is about (more precisely, see dynamic extent, indefinite scope variables). But nowadays it is quite the exception to rely on dynamic scope. Most functions use lexical scope, and this is the case for eval too.

The eval function evaluates a form in the null lexical environment, and it never has access to the surrounding lexical bindings. As such, it behaves like any regular function.

So, in you example, calling eval on the transformed source code will not work, since arg1 to argnN will probably be unbound (it depends on what your macro does).

In order to have an equivalent form, you have to inject bindings in the transformed code, or expand at a higher level:

(defun expand-square (var)
  (list '* var var))

;; instead of:
(defun foo (x) (eval (expand-square 'x))) ;; x unbound during eval

;; inject bindings
(defun foo (x) (eval `(let ((z ,x)) (expand-square z))))

;; or expand the top-level form
(eval `(defun foo (x) ,(expand-square 'x)))

Note that macros (in Common Lisp) also have access to the lexical environment through &environment parameters in their lambda-list. The use of this environment is implementation dependent, but can be used to access the declarations associated with a variable, for example.

Notice also how in the last example you evaluate the code when defining the function, and not when running it. This is the second thing about macro.

Expansion time

In order to emulate macros you could locally replace a call to a macro by a form that emulates it at runtime (using let to captures all the bindings you want to see inside the expanded code, which is tedious), but then you would miss the useful aspect of macros that is: generating code ahead of time.

The last example above shows how you can quote defun and wrap it in eval, and basically you would need to do that for all functions if you wanted to emulate the preprocessing work done by macros.

The macro system is a way to integrate this preprocessing step in the language in a way that is simple to use.

Conclusion

Macros themselves are a nice way to abstract things when functions can't. For example you can have a more human-friendly, stable syntax that hides implementation details. That's how you define pattern-matching abilities in Common Lisp that make it look like they are part of the language, without too much runtime penalty or verbosity.

They rely on simple term-rewriting functions that are integrated in the language, but you can emulate their behavior either at compile-time or runtime yourself if you want. They can be used to perform different kinds of abstraction that are usually missing or more cumbersome to do in other languages, but are also limited: they don't "understand" code by themselves, they don't give access to all the facilities of the compiler (type propagation, etc.). If you want more you can use more advanced libraries or compiler tools (see deftransform), but macros at least are portable.

coredump
  • 37,664
  • 5
  • 43
  • 77
3

Macros are not just functions whose arguments are unevaluated. Macros are functions between programming languages. In other words a macro is a function whose argument is a fragment of source code of a programming language which includes the macro, and whose value is a fragment of source code of a language which does not include the macro (or which includes it in a simpler way).

In very ancient, very rudimentary, Lisps, before people really understood what macros were, you could simulate macros with things called FEXPRs combined with EVAL. A FEXPR was simply a function which did not evaluate its arguments. This worked in such Lisps only because they were completely dynamically scoped, and the cost of it working was that compilation of such things was not possible at all. Those are two enormous costs.

In any modern Lisp, this won't work at all. You can write a toy version of FEXPRs as a macro (this may be buggy):

(defmacro deffex (fx args &body body)
  (assert (every (lambda (arg)
                   (and (symbolp arg)
                        (not (member arg lambda-list-keywords))))
                 args)
      (args) "not a simple lambda list")
  `(defmacro ,fx ,args
     `(let ,(mapcar (lambda (argname argval)
                     `(,argname ',argval))
                    ',args (list ,@args))
        ,@',body)))

So now we could try to write a trivial binding construct I'll call with using this thing:

(deffex with (var val form)
  (eval `(let ((,var ,val)) ,form)))

And this seems to work:

> (with a 1 a)
1

Of course, we're paying the cost that no code which uses this construct can ever be compiled so all our programs will be extremely slow, but perhaps that is a cost we're willing to accept (it's not, but never mind).

Except, of course, it doesn't work, at all:

> (with a 1
    (with b 2
      (+ a b)))

Error: The variable a is unbound.

Oh dear.

Why doesn't it work? It doesn't work because Common Lisp is lexically scoped, and eval is a function: it can't see the lexical bindings.

So not only does this kind of approach prevent compilation in a modern Lisp, it doesn't work at all.

People often, at this point, suggest some kind of kludge solution which would allow eval to be able to see lexical bindings. The cost of such a solution is that all the lexical bindings need to exist in compiled code: no variable can ever be compiled away, not even its name. That's essentially saying that no good compilers can ever be used, even for the small part of your programs you can compile at all in a language which makes extensive use of macros like CL. For instance, if you ever use defun you're not going to be able to compile the code in its body. People do use defun occasionally, I think.

So this approach simply won't work: it worked by happenstance in very old Lisps but it can't work, even at the huge cost of preventing compilation, in any modern Lisp.

More to the point this approach obfuscates the understanding of what macros are: as I said at the start, macros are functions between programming languages, and understanding that is critical. When you are designing macros you are implementing a new programming language.

ignis volens
  • 7,040
  • 2
  • 12
  • That's super interesting - thank you for this interesting answer with FEXPRs! In fact R uses FEXPRs - and in order to imitate Lisp macros, I tried exactly this eval thin in R. Good to get the catch. Actually R is also lexically scoped. – Gwang-Jin Kim Dec 11 '22 at 18:21