Recursive async function that borrows mutable variable twice

Question

I'm designing a connection retry function. It retries if the error was ConnectionClosed. In the full implementation it has other things that justify it being a recursion, but this is off topic. Anyways, since it's a connection, it makes sense to be async. However, async recursives force me to use BoxFuture, and we know that asyncs capture and return all the arguments.

use std::future::Future;
use futures::future::{BoxFuture, FutureExt};

struct Message;

struct Client;

enum Error {
    ConnectionClosed
}

impl Client {
    fn send_and_expect<'a>(
        &'a mut self,
        message: &'a Message
    ) -> BoxFuture<'a, Result<(), Error>> {
        async move {
            Ok(())
        }.boxed()
    }
    
    //With this function I can wrap any async function f that grabs data from the internet
    pub fn connection_retrier<'a, T>(
        f: fn(&'a mut Self, &'a Message) -> T,
        f_self: &'a mut Self,
        f_m: &'a Message,
    ) -> BoxFuture<'a, Result<(),Error>>
    where
        T: Future<Output = Result<(), Error>> + 'a + Send
    {
        /*
            By making f: fn(&'a mut Self, &'a Message) -> T, we tell the compiler that
            `f` is a specific function: one that specifically requires the `Self` and `Message` 
            arguments to live as long as `'a`, where `'a` is the min of the lifetimes of the `f_self` and `f_m`
            arguments passed to `connection_retrier`.
            Thus this forces `f_self` and `f_m` to live as long as `connection_retrier`.
            The call `let r = f(f_self, f_m).await` returns a `Future` that lives as long as `'a`.
            I think this is the problem. The result of the first call borrows `f_self` and `f_m`
            for at least the entire lifetime of `r`. This would make it impossible to use
            `f_self` and `f_m` again inside `connection_retrier`. As you see, I tried making sure
            that `r` destructs before we use `f_self` in `connection_retrier` again, but somehow
            `r` is marked to live as long as `connection_retrier`, because `f_self` is still considered
            to be borrowed.
        */
        async move {
            let ok: bool; 
            {
                let r = f(f_self, f_m).await;
                match r {
                    Ok(_) => ok = true,
                    Err(Error::ConnectionClosed) => {
                        ok = false;
                    }
                }
            }
            match ok {
                true => Ok(()),
                false => Client::connection_retrier(f, f_self, f_m).await
            }
        }.boxed()
    }
    
    async fn send_with_retry<'a> (
        &'a mut self,
        message: &'a Message,
    ) -> Result<(), Error>
    {
        Client::connection_retrier(
            Client::send_and_expect,
            self,
            message
        ).await
    }
}

Error:

error[E0499]: cannot borrow `*f_self` as mutable more than once at a time
  --> src/lib.rs:58:56
   |
24 |         f: fn(&'a mut Self, &'a Message) -> T,
   |         - lifetime `'1` appears in the type of `f`
...
48 |                 let r = f(f_self, f_m).await;
   |                         --------------
   |                         | |
   |                         | first mutable borrow occurs here
   |                         argument requires that `*f_self` is borrowed for `'1`
...
58 |                 false => Client::connection_retrier(f, f_self, f_m).await
   |                                                        ^^^^^^ second mutable borrow occurs here

Playground

I understand why the error occurs: Client::connection_retrier(f, f_self, f_m).await, let's call it r, holds a mutable reference to f_self, so I cannot use it again while it's being held. However, after I check that this result r is Error::ConnectionClosed, I don't need it anymore, so there should be a way to discard it so I can reborrow it mutably.

Answer 1

The reason lies within these three lines:

pub fn connection_retrier<'a, T>(
    f: fn(&'a mut Self, &'a Message) -> T,
    f_self: &'a mut Self,

For f, you are telling the compiler that it will receive a mutable reference with a lifetime equal to that of the mutable reference that connection_retrier receives for f_self (in this case it is 'a).

When you change f's signature to f: fn(&mut Self, &Message) -> T, you are letting the compiler determine the lifetime, which it correctly does by assigning it a lifetime shorter than 'a and hence why it compiles.

Answer 2

The problem

As @moy2010 said, the problem is here:

pub fn connection_retrier<'a, T>(
        f: fn(&'a mut Self, &'a Message) -> T,
        f_self: &'a mut Self,
        f_m: &'a Message,
    ) -> BoxFuture<'a, Result<(), Error>>
where T: Future<Output = Result<(), Error>> + 'a + Send

This signature defines f to be a function that accepts arguments with exactly the lifetime 'a, which is alive throughout the body of connection_retrier.

When you are able to temporarily pass a mutable reference to a function and then 'recover' it later, this is called a reborrow. Reborrowing from a mutable reference works by creating a new mutable reference of shorter lifetime to the data it points to, and preventing use of the original reference until this lifetime has ended. Usually a reborrow happens during a function call, in which case that lifetime is just long enough for the function call to be made.

As you can see, reborrowing always produces a reference with lifetime shorter than the original. But f needs a reference whose lifetime is exactly 'a, so the compiler correctly deduces that the call to f cannot reborrow f_self and instead, f_self is permanently moved into f. Hence the error when you attempt to use it again.

Solutions

If you can change the signature of send_and_expect so that the returned future only borrows from message (i.e. not the Client), then @moy2010 has already provided a correct solution in the comments to their answer. I will proceed on the assumption that send_and_expect must borrow from both its arguments, as indicated by the signature in the question.

Since you would like to reborrow f_self before passing it to f, what you really need is to define f to be a function that can be called for any lifetime (not just 'a). You might think that means we can just change the declaration of f to for<'b> fn(&'b mut Self, &'b Message) -> T. But wait! Your where bound specifies that T borrows from the lifetime 'a, not 'b, so this signature doesn't line up with that of send_and_expect. But also notice that we can't change the bound to say 'b instead, because 'b is not in scope outside of the declaration of f! So there is no way to express the correct bound on T here. There are, however, a few workarounds.

Use a Boxed trait object

The simplest solution is just to make the return type of f a boxed trait object instead of a generic parameter, like this:

pub fn connection_retrier<'a>(
        f: for<'b> fn(&'b mut Self, &'b Message) -> BoxFuture<'b, Result<(), Error>>,
        f_self: &'a mut Self,
        f_m: &'a Message,
    ) -> BoxFuture<'a, Result<(), Error>>

Now we can specify what the return value borrows from inline, avoiding the problem that 'b isn't in scope for a where bound. Since send_and_expect already returns a BoxFuture anyway in your example, this may be the best solution for you.

Use a trait implemented on a ZST instead of a fn pointer

Resorting to boxing and dynamic dispatch to satisfy a compile-time issue might strike some as distasteful, and there is a small performance penalty. This is a trick that can be used to make this work with static dispatch, at the cost of some verbosity. We will have to define our own trait, ConnectFn:

trait ConnectFn<'a> {
    type ConnectFuture: Future<Output=Result<(), Error>> + Send + 'a;
    fn connect(&self, client: &'a mut Client, message: &'a Message) -> Self::ConnectFuture;
}

Then, we can make the signature of connection_retrier the following:

pub fn connection_retrier<'a, F>(
        f: F, 
        f_self: &'a mut Self, 
        f_m: &'a Message,
    ) -> BoxFuture<'a, Result<(), Error>>
where F: for<'b> ConnectFn<'b>

And replace f(f_self, f_m) in the body with f.connect(f_self, f_m). This avoids the need for the return type of f to appear anywhere in connection_retrier's signature, circumventing the issue we had before.

We can then replace our function send_and_expect with a zero-sized type SendAndExpect which implements ConnectFn, like so:

struct SendAndExpect;
impl<'a> ConnectFn<'a> for SendAndExpect
{
    type ConnectFuture = BoxFuture<'a, Result<(), Error>>;
    fn connect(&self, client: &'a mut Client, message: &'a Message) -> Self::ConnectFuture {
        /* the body of Client::send_and_expect goes here. `self` is ignored */
    }
}

With this, we can call connection_retrier like so:

Client::connection_retrier(SendAndExpect, &mut client, &message)

Which is pretty nice. The definition of SendAndExpect itself can be made even nicer, close to that of a regular fn, using macros, but this is left as an exercise to the reader.

Sidenote: You might think that it would be possible to have a blanket implementation of ConnectFn on suitable fn pointers and avoid the need for a new type, but sadly you would be wrong. You'll either run into the same issue with impossible where clause bounds or trait solver limitations when dealing with universally quantified types.