lifetime error when calling method with mutable `self` reference

Question

Here is the code in question:

struct CodeGenerator;
struct Command<'a> {
    command: &'a String,
}
struct Expression;

impl CodeGenerator {
    fn compile_push_arguments<'a>(
        &'a mut self,
        arguments: &'a Vec<Expression>,
    ) -> Vec<Command<'a>> {
        arguments
            .into_iter()
            .flat_map(|argument| self.compile_expression(argument))
            .collect()
    }

    fn compile_expression<'a>(&'a mut self, expression: &'a Expression) -> Vec<Command<'a>> {
        todo!()
    }
}

This produces the following compile error:

error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
  --> src/main.rs:14:39
   |
14 |             .flat_map(|argument| self.compile_expression(argument))
   |                                       ^^^^^^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the lifetime `'_` as defined here...
  --> src/main.rs:14:23
   |
14 |             .flat_map(|argument| self.compile_expression(argument))
   |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that closure can access `self`
  --> src/main.rs:14:34
   |
14 |             .flat_map(|argument| self.compile_expression(argument))
   |                                  ^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined here...
  --> src/main.rs:8:31
   |
8  |     fn compile_push_arguments<'a>(
   |                               ^^
note: ...so that the types are compatible
  --> src/main.rs:12:9
   |
12 | /         arguments
13 | |             .into_iter()
14 | |             .flat_map(|argument| self.compile_expression(argument))
15 | |             .collect()
   | |______________________^
   = note: expected `Vec<Command<'a>>`
              found `Vec<Command<'_>>`

For more information about this error, try `rustc --explain E0495`.

There are a number of things I don't understand in this error message:

• Where is the "autoref" happening here?
• Which lifetime is being referred to when it says "first, the lifetime cannot outlive the lifetime '_ as defined here..."? Does the closure implicitly create a lifetime for itself?
• Where it says "so that the types are compatible" - In what way exactly are the types not currently compatible?

Basically, I want to understand the error.

I noticed that removing the mut from the reference to self fixes it, but I'm not sure why.

Answer 1

Where is the "autoref" happening here?

Autoref happens when you invoke a method. See this question for more information.

I noticed that removing the mut from the reference to self fixes it, but I'm not sure why.

This is actually the problem. Your other questions kind of skirt around it, so I'm going to table them for a moment to explain why omitting mut makes it all work.

Recall that returning a reference from a function extends the borrow given to the function. This is obvious when the types of references match:

fn a<'x>(foo: &'x i32) -> &'x i32 { todo!(); }
fn b<'x>(foo: &'x mut i32) -> &'x mut i32 { todo!(); }

We know that whatever borrow we give the function is extended through the return value. It's less obvious what happens when the mutability of the references disagree:

fn c<'x>(foo: &'x mut i32) -> &'x i32 { todo!(); }

A naive conclusion is that the input borrow is "downgraded" but that isn't the case -- the mutable borrow is extended through the immutable reference that is returned. Here is a concrete example:

fn downgrade<'x>(foo: &'x mut i32) -> &'x i32 { foo }

fn main() {
    let mut i = 0i32;
    let i_ref = downgrade(&mut i);
    let i_ref_2 = downgrade(&mut i);
    
    println!("{} {}", i_ref, i_ref_2);
}

You might expect this to compile, but it does not. i_ref extends the mutable borrow, which prevents i from being borrowed mutably (or even immutably!) a second time.

error[E0499]: cannot borrow `i` as mutable more than once at a time
 --> src/main.rs:6:29
  |
5 |     let i_ref = downgrade(&mut i);
  |                           ------ first mutable borrow occurs here
6 |     let i_ref_2 = downgrade(&mut i);
  |                             ^^^^^^ second mutable borrow occurs here
7 |     
8 |     println!("{} {}", i_ref, i_ref_2);
  |                       ----- first borrow later used here

Once you understand that this doesn't compile, it should be more apparent why your code doesn't compile with mut. Look at the declaration of compile_expression:

fn compile_expression<'a>(
    &'a mut self,
    expression: &'a Expression
) -> Vec<Command<'a>>
{ todo!(); }

This is doing exactly the same thing, but it's obfuscated a bit. Vec<Command<'a>> refers to 'a which means it extends the mutable borrow of self. That's a problem for the closure given to flat_map() because it might be called more than once -- the Vec<Command<'a>> returned from the prior iteration needs to be dropped before the closure can be invoked again. It's effectively the same basic problem as in the simpler example I gave above, but with a more confusing diagnostic from the compiler. (Note this would be an issue even without the .collect() invocation. This closure, when given to flat_map(), creates a lifetime situation that the Iterator trait cannot express. It's akin to the "owning iterator" problem.)

Removing mut makes it work since the borrow extended by the closure isn't exclusive.