What's causing this lifetime error and why can boxing fix it?

Question

I have came across a lifetime quirk that I can't understand. I am unsure how to phrase the problem so it has been difficult finding a solution by googling.

I have a function which creates an instance of a trait, and that returned impl has a lifetime bound 'a. The function takes two parameters (not_borrowed: T, borrowed: &'a str), where not_borrowed is only expected to be available in the body of the function and borrowed has a lifetime bound of 'a meaning it could be stored in the returned trait.

In this example I just have a dummy trait implemented for ().

When I then call the function from an outer function the compiler gives an error:

error[E0597]: `val` does not live long enough
  --> src/main.rs:9:28
   |
8  | fn outer<'a>(val: String) -> impl ReturnedTrait + 'a {
   |          -- lifetime `'a` defined here
9  |     let ret = create_trait(&val, "");
   |               -------------^^^^-----
   |               |            |
   |               |            borrowed value does not live long enough
   |               argument requires that `val` is borrowed for `'a`
...
13 | }
   | - `val` dropped here while still borrowed

I would not expect this to happen because I never required that T lives for 'a.

I noticed that I can get the code to compile by doing what's on the commented line below. I'm not sure why the compiler would all of a sudden change it's mind and think it's OK just because I put the value in a box?

playground

trait ReturnedTrait {}
impl<T: ReturnedTrait + ?Sized> ReturnedTrait for Box<T> {}

impl ReturnedTrait for () {}

fn create_trait<'a, T>(not_borrowed: T, borrowed: &'a str) -> impl ReturnedTrait + 'a {}

fn outer<'a>(val: String) -> impl ReturnedTrait + 'a {
    let ret = create_trait(&val, "");
    // Uncomment this line and the code compiles
    // let ret: Box<dyn ReturnedTrait + 'a> = Box::new(ret);
    ret
}

Answer 1

Here's a simplified version of your code, and a few variations, which should help to explain what's going on (Rust Playground link):

#![feature(type_alias_impl_trait)]
#![feature(unsize)]

use core::marker::Unsize;
use core::fmt::Debug;

pub fn returns_unrelated_impl<T>() -> impl Debug + 'static { () }

returns_unrelated_impl is a simplified version of create_trait. It turns out that it doesn't matter that the function has any arguments at all – all that matters is the generic type parameter (here T as in your example, although the same thing happens with implicit type parameters from impl Trait in argument position).

pub fn test_1<'a>() -> impl Debug {
    returns_unrelated_impl::<&'a ()>()
}

This first test, without a Box, fails for the same reason as in your example (and the same reason as in the post linked in the first comment). Rust has a requirement that an impl Trait in return position should be variant with respect to the generic parameters of the same function, even though this isn't actually necessary to make the type system sound. (There's some potential motivation for this in the linked post.)

However, because this variance requirement isn't required for soundness, the compiler only actually checks it in situations where the type-checker needs to check for variance. Here's an equivalent of the above code using Box<dyn debug>, in which the return value of returns_unrelated_impl is required to be an impl Unsize<dyn Debug> + 'static:

pub fn test_2a<'a>() -> impl Debug {
    type ImplUnsizeStatic = impl Unsize<dyn Debug> + 'static;

    let rv1: ImplUnsizeStatic = returns_unrelated_impl::<&'a ()>();
    let rv2: Box<dyn Debug> = Box::new(rv1);
    rv2
}

This test fails for the same reason as test_1 does: the compiler tries to check whether the impl Debug + 'static variant on T is an impl Unsize<dyn Debug> + 'static that doesn't mention T, and sees that the variance mismatches (even though the requirement for the variance on T is entirely artificial). Changing the code even slightly causes it to now compile:

pub fn test_2b<'a>() -> impl Debug {
    fn verify_impl_unsize_static<T: Unsize<dyn Debug> + 'static>(x: &T) {}

    let rv1 = returns_unrelated_impl::<&'a ()>();
    verify_impl_unsize_static(&rv1);
    let rv2: Box<dyn Debug> = Box::new(rv1);
    rv2
}

test_2a checked whether rv1 was an impl Unsize<dyn Debug> + 'static, whereas test_2b checks whether rv1 implements Unsize<dyn Debug> + 'static. These two statements seem almost identical, but the compiler is interpreting them differently; in test_1 and test_2a I'm trying to coerce an impl Trait type to another impl Trait type and the compiler notices that T is no longer captured, whereas test_2b is simply checking whether the impl Trait type has an appropriate type to implement Unsize<dyn Debug> with a 'static lifetime, and it does.

The reason that boxing fixes the original problem is the same reason that test_2b works: the requirement to coerce a Box<T> to a Box<dyn Debug> is that T implements Unsize<dyn Debug> + 'static, so the coercion to Box<dyn Debug> is checking precisely the same thing that verify_impl_unsize_static does – it's checking to see whether the return value of returns_unrelated_impl implements a trait rather than whether it can be coerced to a given impl type, and the artificial requirement to capture the generic type parameter seems to be checked only in the latter case.

For what it's worth, I consider this behaviour very surprising: it doesn't make too much sense from a programmer's point of view that test_2a and test_2b behave differently, and this makes me suspect that the intended requirement for an impl Trait type in return position to capture the function's generic parameters has been implemented only partially/incompletely. Although the examples above probably explain the current behaviour, I would be very surprised if the current behaviour were what the authors of the language actually intended the compiler to do.