Is there a way to pass a reference to a generic function and return an impl Trait that isn't related to the argument's lifetime?

Question

I've worked down a real-life example in a web app, which I've solved using unnecessary heap allocation, to the following example:

// Try replacing with (_: &String)
fn make_debug<T>(_: T) -> impl std::fmt::Debug {
    42u8
}

fn test() -> impl std::fmt::Debug {
    let value = "value".to_string();

    // try removing the ampersand to get this to compile
    make_debug(&value)
}

pub fn main() {
    println!("{:?}", test());
}

As is, compiling this code gives me:

error[E0597]: `value` does not live long enough
  --> src/main.rs:9:16
   |
5  | fn test() -> impl std::fmt::Debug {
   |              -------------------- opaque type requires that `value` is borrowed for `'static`
...
9  |     make_debug(&value)
   |                ^^^^^^ borrowed value does not live long enough
10 | }
   | - `value` dropped here while still borrowed

I can fix this error in at least two ways:

Instead of passing in a reference to value in test(), pass in value itself
Instead of the parameter T, explicitly state the type of the argument for make_debug as &String or &str

My understanding of what's happening is that, when there is a parameter, the borrow checker is assuming that any lifetime on that parameter affects the output impl Debug value.

Is there a way to keep the code parameterized, continue passing in a reference, and get the borrow checker to accept it?

I don't see the link between your real case and your question, `send_form()` take a `&T`. BTW, see https://stackoverflow.com/questions/55576425/why-does-clippy-suggests-passing-an-arc-as-a-reference — Stargateur, Jul 10 '19 at 16:38
In the real life example, I'm passing a reference to a data structure which is the parameter `T`. I want that data structure inside the reference to itself contain references, which is where the issue is coming from. The presence or absence of the outer reference seems unimportant, but I could be making a mistake. — Michael Snoyman, Jul 10 '19 at 16:53
the link point to the construct of `VerifyRequest ` that doesn't have generic parameter https://github.com/snoyberg/sortasecret/blob/7b44c164181af5267febbb65085954911f3e12c0/src/server.rs#L60 you are very unclear. — Stargateur, Jul 10 '19 at 17:03
This change to the code demonstrates the same error message. I did not claim that the code I linked to was broken, I pointed out that I was using unnecessary allocations to work around the problem. https://github.com/snoyberg/sortasecret/commit/bb43f336e53da6e6437a2f0310785163ab82dee9 — Michael Snoyman, Jul 10 '19 at 17:14
@bitemyapp's answer explains the mechanism by which this is happening, and I've put a hacky workaround in place in my codebase that avoids the clone itself https://github.com/snoyberg/sortasecret/commit/148344046b11869c137ad3cae5628fa95461f361#diff-4ce93534efc34e923ce01e975eb7ed80R106 — Michael Snoyman, Jul 11 '19 at 05:15

bitemyapp · Accepted Answer · 2019-07-11T04:52:19.453

I think this is due to the rules around how impl trait opaque types capture lifetimes.

If there are lifetimes inside an argument T, then an impl trait has to incorporate them. Additional lifetimes in the type signature follow the normal rules.

For more information please see:

A more complete answer

Original goal: the send_form function takes an input parameter of type &T which is rendered to a binary representation. That binary representation is owned by the resulting impl Future, and no remnant of the original &T remains. Therefore, the lifetime of &T need not outlive the impl Trait. All good.

The problem arises when T itself, additionally, contains references with lifetimes. If we were not using impl Trait, our signature would look something like this:

fn send_form<T>(self, data: &T) -> SendFormFuture;

And by looking at SendFormFuture, we can readily observe that there is no remnant of T in there at all. Therefore, even if T has lifetimes of its own to deal with, we know that all references are used within the body of send_form, and never used again afterward by SendFormFuture.

However, with impl Future as the output, we get no such guarantees. There's no way to know if the concrete implementation of Future in fact holds onto the T.

In the case where T has no references, this still isn't a problem. Either the impl Future references the T, and fully takes ownership of it, or it doesn't reference it, and no lifetime issues arise.

However, if T does have references, you could end up in a situation where the concrete impl Future is holding onto a reference stored in the T. Even though the impl Future has ownership of the T itself, it doesn't have ownership of the values referenced by the T.

This is why the borrow check must be conservative, and insist that any references inside T must have a 'static lifetime.

The only workaround I can see is to bypass impl Future and be explicit in the return type. Then, you can demonstrate to the borrow checker quite easily that the output type does not reference the input T type at all, and any references in it are irrelevant.

The original code in the actix web client for send_form looks like:

https://docs.rs/awc/0.2.1/src/awc/request.rs.html#503-522

pub fn send_form<T: Serialize>(
        self,
        value: &T,
    ) -> impl Future<
        Item = ClientResponse<impl Stream<Item = Bytes, Error = PayloadError>>,
        Error = SendRequestError,
    > {
        let body = match serde_urlencoded::to_string(value) {
            Ok(body) => body,
            Err(e) => return Either::A(err(Error::from(e).into())),
        };

        // set content-type
        let slf = self.set_header_if_none(
            header::CONTENT_TYPE,
            "application/x-www-form-urlencoded",
        );

        Either::B(slf.send_body(Body::Bytes(Bytes::from(body))))
    }

You may need to patch the library or write your own function that does the same thing but with a concrete type. If anyone else knows how to deal with this apparent limitation of impl trait I'd love to hear it.

Here's how far I've gotten on a rewrite of send_form in awc (the actix-web client library):

    pub fn send_form_alt<T: Serialize>(
        self,
        value: &T,
        // ) -> impl Future<
        //     Item = ClientResponse<impl Stream<Item = Bytes, Error = PayloadError>>,
        //     Error = SendRequestError,
    ) -> Either<
        FutureResult<String, actix_http::error::Error>,
        impl Future<
            Item = crate::response::ClientResponse<impl futures::stream::Stream>,
            Error = SendRequestError,
        >,
    > {

Some caveats so far:

Either::B is necessarily an opaque impl trait of Future.
The first param of FutureResult might actually be Void or whatever the Void equivalent in Rust is called.

The first issue you link to is closed and apparently fixed. Why do you believe it is relevant to today's code? — Shepmaster, Jul 10 '19 at 16:09
@Shepmaster background flavor for the design. I anticipated getting this comment and people might misunderstand so I'll remove. My goal ultimately when poking at stuff like this is to develop enough intuition for the design that I don't need to google compiler errors. Seeing how past bugs were fixed (that is, behaviors at variance with the intended design) can tell me a lot. I get the impression SO isn't a place that likes a Talmudic approach so I redacted the reference. — bitemyapp, Jul 10 '19 at 16:14
I totally think that it's reasonable to provide background information, but providing links without context isn't useful. Introduce the link and explain the basic content. However, it's true that most people want an answer to "fix my code now", so it's often useful putting a TL;DR at the top with the copy-pastable solution, followed by the deeper explanation. — Shepmaster, Jul 10 '19 at 16:18
It's also good to directly answer the OPs question: "how do I do this". For whatever human reason, we tend to put our actual point at the end of our questions; I've retitled the question to put the last sentence first. — Shepmaster, Jul 10 '19 at 16:21
@Shepmaster I hear you. In this particular SO question, I'm replying to my boss. What would a quick fix for this be? "Don't move the value without explicit lifetimes or borrow the value?" Seems like an issue that needs clarifying, with the "answer" being to understand how the error can be triggered or avoided. I demonstrated both. This is a tiny repro of the original issue from the actix-web app he's working on. — bitemyapp, Jul 10 '19 at 16:22
I see now there's, "Is there a way to keep the code parameterized, continue passing in a reference, and get the borrow checker to accept it?" I'm not sure if you edited that in or not. I'm also unsure if what I demonstrated constitutes a fix or not. Seems too trivial to be the case for the repro but maybe I missed something. — bitemyapp, Jul 10 '19 at 16:28
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/196269/discussion-between-bitemyapp-and-shepmaster). — bitemyapp, Jul 10 '19 at 16:29

Is there a way to pass a reference to a generic function and return an impl Trait that isn't related to the argument's lifetime?

1 Answers1

A more complete answer