27

So I've got the following macro code I'm trying to debug. I've taken it from the Rust Book under the section "The deep end". I renamed the variables within the macro to more closely follow this post.

My goal is to have the program print out each line of the BCT program. I'm well aware that this is very compiler heavy.

The only error rustc is giving me is:

user@debian:~/rust/macros$ rustc --pretty expanded src/main.rs -Z unstable-options > src/main.precomp.rs
src/main.rs:151:34: 151:35 error: no rules expected the token `0`
src/main.rs:151     bct!(0, 1, 1, 1, 0, 0, 0; 1, 0);

What steps can I take to figure out where in the macro the problem is coming from?

Here's my code:

fn main() {
{
    // "Bitwise Cyclic Tag" automation through macros
    macro_rules! bct {
        // cmd 0:  0 ... => ...
        (0, $($program:tt),* ; $_head:tt)
            => (bct_p!($($program),*, 0 ; ));
        (0, $($program:tt),* ; $_head:tt, $($tail:tt),*)
            => (bct_p!($($program),*, 0 ; $($tail),*));

        // cmd 1x:  1 ... => 1 ... x
        (1, $x:tt, $($program:tt),* ; 1)
            => (bct_p!($($program),*, 1, $x ; 1, $x));
        (1, $x:tt, $($program:tt),* ; 1, $($tail:tt),*)
            => (bct_p!($($program),*, 1, $x ; 1, $($tail),*, $x));

        // cmd 1x:  0 ... => 0 ...
        (1, $x:tt, $($program:tt),* ; $($tail:tt),*)
            => (bct_p!($($program),*, 1, $x ; $($tail),*));

        // halt on empty data string
        ( $($program:tt),* ; )
            => (());
        }

    macro_rules! print_bct {
        ($x:tt ; )
            => (print!("{}", stringify!($x)));
        ( ; $d:tt)
            => (print!("{}", stringify!($d)));
        ($x:tt, $($program:tt),* ; )
            => {
                print!("{}", stringify!($x));
                print_bct!($program ;);
            };
        ($x:tt, $($program:tt),* ; $($data:tt),*)
            => {
                print!("{}", stringify!($x));
                print_bct!($program ; $data);
            };
        ( ; $d:tt, $($data:tt),*)
            => {
                print!("{}", stringify!($d));
                print_bct!( ; $data);
            };
    }

    macro_rules! bct_p {
        ($($program:tt),* ; )
            => {
                print_bct!($($program:tt),* ; );
                println!("");
                bct!($($program),* ; );
            };
        ($($program:tt),* ; $(data:tt),*)
            => {
                print_bct!($($program),* ; $($data),*);
                println!("");
                bct!($($program),* ; $($data),*);
            };
    }

    // the compiler is going to hate me...
    bct!(0, 1, 1, 1, 0, 0, 0; 1, 0);
}
Lukas Kalbertodt
  • 79,749
  • 26
  • 255
  • 305
Nashenas
  • 1,651
  • 1
  • 21
  • 25
  • Possible duplicate of [How do I see the expanded macro code that's causing my compile error?](https://stackoverflow.com/q/28580386/1804173) – bluenote10 Feb 11 '23 at 14:18

3 Answers3

47

There's two main ways to debug macros that are failing to expand:

  • trace_macros! and
  • log_syntax!

(NB. both are feature gated, under features of the same name, and so require the nightly compiler to work, rustup makes it easy to switch between versions for this sort of work.)

trace_macros!(...) takes a boolean argument that switches macro tracing on or off (i.e. it's stateful), if it's on, the compiler will print each macro invocation with its arguments as they are expanded. Usually one just wants to throw a trace_macros!(true); call at the top of the crate, e.g. if one adds the following to the top of your code:

#![feature(trace_macros)]

trace_macros!(true);

Then the output looks like:

bct! { 0 , 1 , 1 , 1 , 0 , 0 , 0 ; 1 , 0 }
bct_p! { 1 , 1 , 1 , 0 , 0 , 0 , 0 ; 0 }
<anon>:68:34: 68:35 error: no rules expected the token `0`
<anon>:68     bct!(0, 1, 1, 1, 0, 0, 0; 1, 0);
                                           ^
playpen: application terminated with error code 101

which hopefully narrows down the problem: the bct_p! call is invalid in some way. Looking at it carefully reveals the problem, the left-hand side of second arm of bct_p uses data:tt when it should use $data:tt, i.e. a missing $.

    ($($program:tt),* ; $(data:tt),*)

Fixing that allows compilation to make progress.

log_syntax! isn't as immediately useful in this case, but is still a neat tool: it takes arbitrary arguments and prints them out when it is expanded, e.g.

#![feature(log_syntax)]

log_syntax!("hello", 1 2 3);

fn main() {}

will print "hello" , 1 2 3 as it compiles. This is most useful to inspect things inside other macro invocations.

(Once you've got expansion to work, the best tool to debug any problems in the generated code is to use the --pretty expanded argument to rustc. NB. this requires the -Z unstable-options flag to be passed to activate it.)

John Vandenberg
  • 474
  • 6
  • 16
huon
  • 94,605
  • 21
  • 231
  • 225
  • Why doesn't the macro compilation step complain about `$(not_al_variable),*` ? In what situations would that be valid on its own? Also, don't forget about --pretty expanded,hygiene, which is useful when a variable outside the macro has the same name as a variable inside the macro (at least, that's how it was explained to me). – Nashenas May 13 '15 at 12:56
  • 2
    It might make sense to eat a lot of copies of an exact thing, e.g. one could strip trailing zeros with `$(0),*` which would match only `0, 0, 0` (etc.). That said, seems relatively rare that this would be hugely useful. – huon May 14 '15 at 00:35
24

Another great tool to use for easily seeing the expansion is cargo-expand.

It can be installed with:

cargo install cargo-expand

And then used quite simply as:

cargo expand

Or with more precision to target a particular test file (tests/simple.rs for example):

cargo expand --test simple

Be sure to check out the --help, there are a bunch of options to narrow down what is expanded. You can even target individual items (structs, fns etc.) for expansion!

Zefira
  • 4,329
  • 3
  • 25
  • 31
0

Debugging was interesting. I started with the simplest possible input, and worked up from there. I found that I had issues along the way with the printing functions (rewrite so it just prints inputs and doesn't cycle back!).

I also added rules that were more explicit, and then removed them once everything was working (one by one of course, testing along the way). Once I knew each individual piece was compiling, and the printing functions were working, I was able to verify the output of the macros. The below macro sometimes runs when it shouldn't, but it compiles, and prints and is debuggable. I'm happy enough with the current state to post it here.

fn main() {
    // "Bitwise Cyclic Tag" automation through macros
    macro_rules! bct {
        // cmd 0:  0 ... => ...
        (0, $($program:tt),* ; $_head:tt)
            => (pbct!($($program),*, 0 ; ));
        (0, $($program:tt),* ; $_head:tt, $($tail:tt),*)
            => (pbct!($($program),*, 0 ; $($tail),*));

        // cmd 1x:  1 ... => 1 ... x
        (1, $x:tt, $($program:tt),* ; 1)
            => (pbct!($($program),*, 1, $x ; 1, $x));
        (1, $x:tt, $($program:tt),* ; 1, $($tail:tt),*)
            => (pbct!($($program),*, 1, $x ; 1, $($tail),*, $x));

        // cmd 1x:  0 ... => 0 ...
        (1, $x:tt, $($program:tt),* ; $($tail:tt),*)
            => (pbct!($($program),*, 1, $x ; $($tail),*));

        // halt on empty data string
        ( $($program:tt),* ; )
            => (());
    }

    macro_rules! println_bct {
        () =>
            (println!(""));
        (;) =>
            (println!(":"));

        ($d:tt) =>
            (println!("{}", stringify!($d)));
        ($d:tt, $($data:tt),*) => {
            print!("{}", stringify!($d));
            println_bct!($($data),*);
        };
        ( ; $($data:tt),*) => {
            print!(":");
            println_bct!($($data),*);
        };

        ($x:tt ; $($data:tt),*) => {
            print!("{}", stringify!($x));
            println_bct!( ; $($data),*);
        };
        ($x:tt, $($program:tt),* ; $($data:tt),*) => {
            print!("{}", stringify!($x));
            println_bct!($($program),* ; $($data),*);
        };
    }

    macro_rules! pbct {
        ($($program:tt),* ; $($data:tt),*) => {
            println_bct!($($program),* ; $($data),*);
            bct!($($program),* ; $($data),*);
        };
    }

    pbct!(0, 0, 1, 1, 1, 0, 0, 0 ; 1, 0, 1);

    // This one causes the compiler to hit recursion limits, heh
    // pbct!(0, 0, 1, 1, 1, 1, 1, 0 ; 1, 0, 1);
}
Nashenas
  • 1,651
  • 1
  • 21
  • 25