3

I am using a trait with a method that returns an boxed iterator. Since the iterator will use self and the parameters of foo, all are constrained to the same lifetime:

pub trait Foo {
    fn foo<'a>(&'a self, txt: &'a str) -> Box<Iterator<Item = String> + 'a>;
}

I would like to build a function around this method:

fn foo_int<'a, F: Foo>(f: &'a F, val: i32) -> impl Iterator<Item = String> + 'a {
    let txt = format!("{}", val);
    f.foo(&txt)
}

But that does not compile, because:

error[E0515]: cannot return value referencing local variable `txt`
 --> src/lib.rs:7:5
  |
7 |     f.foo(&txt)
  |     ^^^^^^----^
  |     |     |
  |     |     `txt` is borrowed here
  |     returns a value referencing data owned by the current function

I understand why this happens, and it makes sense, but it seems to be that there should be a way to get around it. After all, that's what closures (with the move keyword) do: they take ownership of the values they need to "take away with them".

Is there a clever way to rewrite the foo_int function, using a closure or something else?

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
Pierre-Antoine
  • 1,915
  • 16
  • 25
  • 1
    What does your `foo` method actually do with the `str`? If it needs a string that lives as long as `f`, wouldn't it make sense for `foo` to accept a `String` instead of `&str`? – loganfsmyth Feb 15 '19 at 19:12
  • For the sake of the argument, let's assume I have no control over the trait Foo here. (full disclosure: in my reall situation, I do have control, so I may indeed consider your solution... but I'm expecting this kind of problem in the future with somebody else's trait) – Pierre-Antoine Feb 15 '19 at 19:50
  • If the trait requires that `txt` live just as long as `self`, there is no way to create a _new_ string inside `foo_int` and have it satisfy that lifetime. By definiton, `f` already exists by the time `foo_int` is called, so there is nothing you can do in `foo_int` to create something with that lifetime. – loganfsmyth Feb 15 '19 at 20:04
  • In a context like this, the only `&str` that would satisfy the constraint would be a `&'static str` pretty much. – loganfsmyth Feb 15 '19 at 20:08
  • @Pierre-Antoine did you consider using macros ? – Ömer Erden Feb 15 '19 at 20:20
  • "After all, that's what closure (with the move keyword) do: they take ownership of the values they need to "take away with them"." yeah closure copy the pointer, but the pointer need to refer to something valid. It's impossible to do what you want with this definition of `foo()` – Stargateur Feb 15 '19 at 20:24
  • The first duplicate explains why you cannot, in general, have the `String` and a reference to it (the iterator from `foo`) inside the same struct (the iterator from `foo_int`). It also explains that there are cases where such a thing is possible: as you pointed out, a `String` has a stable memory address. The other two Q&A show various tools in the Rust ecosystem that hide the required `unsafe` code for you. – Shepmaster Feb 15 '19 at 21:14
  • 1
    *After all, that's what closures [...] do* — [closures have the exact same problem](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=349a494afb18dd2ce4665f36f42d34c0). – Shepmaster Feb 15 '19 at 21:20
  • @loganfsmyth "the trait requires that txt live just as long as self" No, not exactly. It requires that both `self` and `txt` are borrowed as long as the returned iterator lives, which is quite different. The total life durations of `self` and `txt` can still be different. As you suggest, one solution is to have `txt` live for the `static` lifetime, even if `self` does not. – Pierre-Antoine Feb 15 '19 at 21:53

1 Answers1

1

Actually, while working my minimal example, I came up with an idea that (kind of) solves the problem. It is available in the playground.

The idea is to wrap the parameter of foo and the returned iterator in a dedicated type, which itself implements the Iterator trait by delegating to the inner iterator.

pub struct Bar<'a>(String, Box<Iterator<Item = String> + 'a>);

impl<'a> Bar<'a> {
    fn new<F: Foo>(foo: &'a F, txt: String) -> Bar<'a> {
        let itr = foo.foo(unsafe { &*(&txt[..] as *const str) });
        Bar(txt, itr)
    }
}

impl<'a> Iterator for Bar<'a> {
    type Item = String;

    fn next(&mut self) -> Option<Self::Item> {
        self.1.next()
    }
}

fn foo_int<'a, F: Foo>(f: &'a F, val: i32) -> impl Iterator<Item = String> + 'a {
    Bar::new(f, format!("{}", val))
}

I'm not entirely satisfied with that solution, though, because of the following reasons.

  • It is rather specific to this particular method; for another method with other parameters, I would need a different wrapper type.

  • It uses unsafe code. I believe this one is ok (because the inner &str of String is stable through move), but it might be harder (if not impossible) to achieve with other types.

  • It forces me to re-implement Iterator. While this minimal version works, it is suboptimal because the underlying iterator may have optimized implementations of some methods, which I "loose" here by relying on the default implementations. Implementing all methods of Iterator by hand (to pass them through to self.1) is tedious, and brittle (if further versions add new methods, I would have to add them as well).

So any better solution is still welcome...

edit: @Shepmaster's arguments convinced me; this is actually the same old problem of storing a value and a reference to it. So it is unsafe, and my solution is not general, because in general, it may not work :-/

Pierre-Antoine
  • 1,915
  • 16
  • 25
  • "So any better solution is still welcome..." I would say better design, here the solution is obvious `foo()`, should take a String, period. – Stargateur Feb 15 '19 at 20:21
  • @Stargateur I respectfully disagree. Any situation where the iterator is consumed in place (e.g. a `for` loop) works perfectly with a `&str` parameters. Forcing the user to copy the data by requiring a `String` would induce a cost that is not strictly required in those situations... – Pierre-Antoine Feb 17 '19 at 17:53
  • I didn't fully understand your use case. But this look like a exception, how rust is suppose to know if the reference is valid ? In C you could do that but not in Rust because it's unsafe so compiler disallow it by default. Also, I don't think in your example, that the cost of one extra copy matter, O(n) to O(n + 1) for safe behavior. also if your function take a String, you can avoid the extra copy if you don't clone the string you give and so you don't have any extra copy. `f.foo(format!("{}", val))` will not perform extra copy. – Stargateur Feb 17 '19 at 18:41
  • @Stargateur granted, *in my minimal example*, the best solution would be to change the signature of `foo`. My real situation is a bit too specific to explain on SO, but please trust me that it makes sense to expect a reference as a parameter. Also, I agree that I'm pushing the Rust compiler outside of its comfort zone. As Shepmaster pointed out, an `unsafe` block is inevitable here. – Pierre-Antoine Feb 17 '19 at 20:10