How do I access the token stream of a trait when writing a derive macro for a struct?

Question

I have a struct and a trait:

struct Foo {
    i: i32,
}

trait FooTrait {
    fn foo(&self);
}

I want to create a derive macro for the struct which generates the impl:

impl FooTrait for Foo {
    fn foo(&self) {
        println!("generated code: {}", self.i);
    }
}

When I attempted to achieve this, I'm facing the blocker that my derive macro doesn't seem to have a way to know the token stream of FooTrait, where I need to iterate through the methods of FooTrait, and generate the implementation for each trait method based on Foo!

How can I achieve that?

This question is not about how to use quote! to quote the trait impl and spell out foo directly - the hard part is that I want to procedurally iterate through the methods of FooTrait, and generate some boilerplate code for each trait method.

The input TokenStream is made up of the source code, the `Foo` struct in your case. I don't think it contains the method information of `FooTrait`, which is what the derive macro needs to be implemented to output via the output `TokenStream` based on the input one. — Joe_Jingyu, Nov 30 '21 at 08:06
[HeapSize](https://github.com/dtolnay/syn/tree/master/examples/heapsize), mentioned in the [Syn](https://docs.rs/syn/latest/syn/) documentation, is a good example to show how the input token stream is meant to be used. — Joe_Jingyu, Nov 30 '21 at 08:13

score 1 · Answer 1 · answered Nov 30 '21 at 17:44

1

You don't.

By construction, macros are only provided with the source code of the item they are attached to. If you attach the macro to the struct, you don't get to see the trait.

Similar derive macros bake in the knowledge about the trait being implemented. They may also parse custom attributes that allow the user to configure details about the generated code. Your macro will likely do the same.

See also:

Is it possible to store state within Rust's procedural macros?

answered Nov 30 '21 at 17:44

Shepmaster

388,571
95
1,107
1,366

This makes proc macros rather limited IMHO. It would be really nice if there is a way to do crate level ast cross reference (like static reflection/introspection) inside a proc macro, so each proc macro would have better context info - if it wants to reference some other ast items within the current macro processing, it could do so. – Dejavu Dec 02 '21 at 03:48
@Dejavu If you have some free time, I'm sure the compiler team would appreciate you adding that support to the compiler. My understanding is that doing so would be very complex. Many good procedural macros exist within the current limitations, but maybe with your work we'd have even more amazing things! – Shepmaster Dec 03 '21 at 14:52

Alexey Romanov · Answer 2 · 2021-12-01T00:15:18.930

0

A possible workaround: instead of defining FooTrait directly, encode the information you need about it in some structure, store it in a const and access it both from the derive macro and another macro generating the definition of FooTrait.

EDIT: if you want exactly "the token stream of FooTrait", you don't need the const (this is just a skeleton and not tested):

The macros crate:

#[proc_macro]
pub fn foo_trait_definition(_input: TokenStream) -> TokenStream {
    TokenStream::from(quote!(
        trait FooTrait {
            ... // actual definition goes here
        }
    ))
}

#[proc_macro_derive(FooTrait)]
pub fn foo_trait_derivation(input: TokenStream) -> TokenStream {
    let foo_trait_tokens: TokenStream = foo_trait_definition(TokenStream::from(quote!()));

    // you can parse foo_trait_tokens and use them just like input

    TokenStream::from(quote!(
        impl foo_mod::FooTrait for #name {
            ...
        }
    ))
}

The trait crate:

mod foo_mod {
    foo_trait_definition!();
}

and

#[derive(FooTrait)]
struct Foo {
    i: i32,
}

But I expect that in most cases the macros crate can look like this instead of parsing foo_trait_tokens:

// lists method names here, if they are all (&self)
// but probably you want something more complex
const foo_trait_data = ["foo"];

#[proc_macro]
pub fn foo_trait_definition(_input: TokenStream) -> TokenStream {
    // use foo_trait_data here
 
    TokenStream::from(quote!(
        trait FooTrait {
            ... // actual definition goes here
        }
    ))
}

#[proc_macro_derive(FooTrait)]
pub fn foo_trait_derivation(input: TokenStream) -> TokenStream {
    // use foo_trait_data here too

    TokenStream::from(quote!(
        impl foo_mod::FooTrait for #name {
            ...
        }
    ))
}

edited Dec 01 '21 at 00:15

answered Nov 30 '21 at 19:56

Alexey Romanov

167,066
35
309
487

How do you propose that the derive macro access the `const`? – Shepmaster Nov 30 '21 at 20:19
I haven't written an example to test, but a derive macro is defined by a function, is there a reason it can't access a const in the same module or another module in the same crate? – Alexey Romanov Nov 30 '21 at 22:07
The answer you linked, https://stackoverflow.com/a/52914046/9204, suggests "you could try using static global variables". That's what I meant, with the variable being immutable; the state doesn't need to change. This removes the concerns about macro invocation ordering or skipped invocations due to incremental compilation. – Alexey Romanov Nov 30 '21 at 22:11
Do you mean to state that the constant is in the crate that *defines* or *uses* the derive macro? – Shepmaster Nov 30 '21 at 23:14
The crate that defines both the derive macro and the define-the-trait macro. I've edited the answer, hopefully it makes the idea more clear. – Alexey Romanov Dec 01 '21 at 00:10
Also: I believe something like this can work, but I haven't actually needed it and so haven't tried; if I am wrong, I'd be happy to learn why. – Alexey Romanov Dec 01 '21 at 00:23
So this answer is an expansion of “derive macros bake in the knowledge about the trait being implemented”? – Shepmaster Dec 01 '21 at 00:31
Yes, but in a specific way which provides "a way to know the token stream of FooTrait" and "to procedurally iterate through the methods of FooTrait" as the question asks. So it's also in opposition to the "you don't" part. – Alexey Romanov Dec 01 '21 at 00:41
Or to put it another way: this approach lets you change `FooTrait` (either through `foo_trait_definition` in the first case or `foo_trait_data` in the second) without changing the derive macro's definition (`foo_trait_derivation`). That's what the question seems to be basically about. – Alexey Romanov Dec 01 '21 at 00:47

How do I access the token stream of a trait when writing a derive macro for a struct?

2 Answers2