How can I include the contents of a file as an argument to a macro?

Question

I'm trying to implement a Lisp version of Processing, and to that end I'm employing the macro_lisp crate to turn Lisp code into Rust at compile time.

It works when I structure my code like so:

main.rs

fn main() {
    include!("hello.lisp");
}

hello.lisp

lisp!(println "hello")

Note that I have to wrap the content of hello.lisp in lisp!(), in hello.lisp itself.

I want it to be structured like so:

main.rs

fn main() {
    lisp!(include!("hello.lisp"));
}

hello.lisp

println "hello"

But this gives me the following error:

error: expected expression, found `<eof>`
  --> src/main.rs:47:18
   |
47 |     lisp!(include!("draw.lisp"));
   |                  ^ expected expression

There should not be an EOF there, there should be hello "list".

What am I doing wrong? Do I need to patch macro_lisp?

Answer 1

Unfortunately, what you want is not easy to achieve.

Macros work significantly different than functions. The important part for this question is that "nested macro calls" are evaluated from "outside to inside" (unlike functions, where arguments are evaluated first, so "inside to outside"). We can see that in effect with this tiny program:

macro_rules! foo {
    ($x:literal) => { "literal" };
    ($x:ident ! ()) => { "ident ! ()" };
}

macro_rules! bar {
    () => { 3 };
}

fn main() {
    let s = foo!(bar!());
    println!("{}", s);
}

As you can see on the Playground, it prints ident ! (). That means the bar!() macro was not evaluated before foo! was evaluated. (Furthermore, the compiler even warns about the unused macro definition bar.)

include! is no exception to this, so we can't use it as an argument to other macros. So how can you do it? I can think of two ways, neither of which is particularly easy or elegant:

• Write a procedural macro that loads the file and emits the token stream lisp! { ... } where ... is the file content. It could be tricky to get all the paths right (and to make sure everything is correctly recompiled when the lisp file changes), but it should in theory work.

• Use a build script to manually replace include!("*.lisp") strings in your source code with the file content. Obviously, you don't actually want to modify your real source code (that is checked into git), but have to be a bit clever about it. A few crates use such a tactic, but only for very special reasons. I wouldn't advise to do this in your case.

In your case, I would think twice whether using a lot of LISP code in Rust is a good idea, because there aren't any nice ways to make it work (as far as I can see).

Answer 2

While this question is now 4 years old, I am the author of a crate that solves this exact issue in a maintainable way, and I'd like to save others the effort of re-implementing the required boilerplate by sharing how this would be solved using the CPS crate.

Lukas' answer highlights that Rust requires some project-external code, like a build script or proc-macro, to implement inside-out macro evaluation. CPS is a crate that does the required proc-macro stuff for you, and introduces let bindings that allow you to write your desired lisp! macro as a macro-by-example:

#[cps::cps]
macro_rules! lisp_include {
    ($source:literal) => 
    let $($lisp_source:tt)* = cps::include!($source) in
    {
        lisp!($($lisp_source)*)
    }
}

After which you can use this extended version of the lisp! macro elsewhere in your code:

lisp_include!("hello.lisp");