124

I want to write a program that will write a file in 2 steps. It is likely that the file may not exist before the program is run. The filename is fixed.

The problem is that OpenOptions.new().write() can fail. In that case, I want to call a custom function trycreate(). The idea is to create the file instead of opening it and return a handle. Since the filename is fixed, trycreate() has no arguments and I cannot set a lifetime of the returned value.

How can I resolve this problem?

use std::io::Write;
use std::fs::OpenOptions;
use std::path::Path;

fn trycreate() -> &OpenOptions {
    let f = OpenOptions::new().write(true).open("foo.txt");
    let mut f = match f {
        Ok(file)  => file,
        Err(_)  => panic!("ERR"),
    };
    f
}

fn main() {
    {
        let f = OpenOptions::new().write(true).open(b"foo.txt");
        let mut f = match f {
            Ok(file)  => file,
            Err(_)  => trycreate("foo.txt"),
        };
        let buf = b"test1\n";
        let _ret = f.write(buf).unwrap();
    }
    println!("50%");
    {
        let f = OpenOptions::new().append(true).open("foo.txt");
        let mut f = match f {
            Ok(file)  => file,
            Err(_)  => panic!("append"),
        };
        let buf = b"test2\n";
        let _ret = f.write(buf).unwrap();
    }
    println!("Ok");
}
Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
Nex
  • 1,423
  • 2
  • 10
  • 7
  • Open this page, Ctrl-F, "Cow", no results?? While you can't return references to variables created in functions, you can use `std::borrow::Cow` to generalize over owned data and unowned references -- it's a `Deref` that lets you choose whether a given instance owns or borrows its data. I've found it to be the most reliable way to switch off between returning owned and unowned data. – BallpointBen Aug 15 '21 at 19:41

5 Answers5

124

The question you asked

TL;DR: No, you cannot return a reference to a variable that is owned by a function. This applies if you created the variable or if you took ownership of the variable as a function argument.

Solutions

Instead of trying to return a reference, return an owned object. String instead of &str, Vec<T> instead of &[T], T instead of &T, etc.

If you took ownership of the variable via an argument, try taking a (mutable) reference instead and then returning a reference of the same lifetime.

In rare cases, you can use unsafe code to return the owned value and a reference to it. This has a number of delicate requirements you must uphold to ensure you don't cause undefined behavior or memory unsafety.

See also:

Deeper answer

fjh is absolutely correct, but I want to comment a bit more deeply and touch on some of the other errors with your code.

Let's start with a smaller example of returning a reference and look at the errors:

fn try_create<'a>() -> &'a String {
    &String::new()
}

Rust 2015

error[E0597]: borrowed value does not live long enough
 --> src/lib.rs:2:6
  |
2 |     &String::new()
  |      ^^^^^^^^^^^^^ temporary value does not live long enough
3 | }
  | - temporary value only lives until here
  |
note: borrowed value must be valid for the lifetime 'a as defined on the function body at 1:15...
 --> src/lib.rs:1:15
  |
1 | fn try_create<'a>() -> &'a String {
  |               ^^

Rust 2018

error[E0515]: cannot return reference to temporary value
 --> src/lib.rs:2:5
  |
2 |     &String::new()
  |     ^-------------
  |     ||
  |     |temporary value created here
  |     returns a reference to data owned by the current function

Is there any way to return a reference from a function without arguments?

Technically "yes", but for what you want, "no".

A reference points to an existing piece of memory. In a function with no arguments, the only things that could be referenced are global constants (which have the lifetime &'static) and local variables. I'll ignore globals for now.

In a language like C or C++, you could actually take a reference to a local variable and return it. However, as soon as the function returns, there's no guarantee that the memory that you are referencing continues to be what you thought it was. It might stay what you expect for a while, but eventually the memory will get reused for something else. As soon as your code looks at the memory and tries to interpret a username as the amount of money left in the user's bank account, problems will arise!

This is what Rust's lifetimes prevent - you aren't allowed to use a reference beyond how long the referred-to value is valid at its current memory location.

See also:

Your actual problem

Look at the documentation for OpenOptions::open:

fn open<P: AsRef<Path>>(&self, path: P) -> Result<File>

It returns a Result<File>, so I don't know how you'd expect to return an OpenOptions or a reference to one. Your function would work if you rewrote it as:

fn trycreate() -> File {
    OpenOptions::new()
        .write(true)
        .open("foo.txt")
        .expect("Couldn't open")
}

This uses Result::expect to panic with a useful error message. Of course, panicking in the guts of your program isn't super useful, so it's recommended to propagate your errors back out:

fn trycreate() -> io::Result<File> {
    OpenOptions::new().write(true).open("foo.txt")
}

Option and Result have lots of nice methods to deal with chained error logic. Here, you can use or_else:

let f = OpenOptions::new().write(true).open("foo.txt");
let mut f = f.or_else(|_| trycreate()).expect("failed at creating");

I'd also return the Result from main. All together, including fjh's suggestions:

use std::{
    fs::OpenOptions,
    io::{self, Write},
};

fn main() -> io::Result<()> {
    let mut f = OpenOptions::new()
        .create(true)
        .write(true)
        .append(true)
        .open("foo.txt")?;

    f.write_all(b"test1\n")?;
    f.write_all(b"test2\n")?;

    Ok(())
}
Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
  • 15
    Note: in C++, returning a reference to a stack-local variable is Undefined Behavior; if it appears to work, you are just unlucky. In common situations, the compilers should detect the problem and emit a warning. – Matthieu M. Sep 21 '15 at 08:15
  • @MatthieuM. just a warning... how unsafe :-) Although it shows that it's been a few years since I wrote C for a day job, as I never got to see any of those warnings. Nice to see progress in all camps! – Shepmaster Sep 21 '15 at 14:20
  • @Shepmaster: You must have been using an old compiler; I'm still stuck on gcc 4.3.2 and I have it! But yes, only a warning. Most C/C++ compilers take a conservative approach: errors are mandated by the Standard, and the rest is done with warnings (with more or less accuracy ...) – Matthieu M. Sep 21 '15 at 14:53
  • @MatthieuM. We certainly had an older compiler, but maybe not that old. It's possible that I never tried to return a local variable as a reference, of course, but I don't know that I could claim to be that good. :-) – Shepmaster Sep 21 '15 at 14:56
  • Could you possibly explain how to use the value returned from Result returned from TryCatch and call the write method? I tried your suggestions but when i unwrap the result using let inside = self.returnedresultsfile.as_ref().unwrap(); and then match to a file it has the wrong type and produces the "error: no method named `Write` found for type `&std::fs::File` in the current scope". I would really appreciate an explanation of how you can call the write method from the returned value :) – D3181 Apr 24 '19 at 19:16
  • 1
    @D3181 there *is* no method called `Write`. You are looking for [`write`](https://doc.rust-lang.org/std/io/trait.Write.html#tymethod.write), and you will need to have the trait in scope. Check out the Rust 1.0 / "write a file" section of [What's the de-facto way of reading and writing files in Rust 1.x?](https://stackoverflow.com/a/31193386/155423). – Shepmaster Apr 24 '19 at 23:25
  • Just want to point out that in _some_ cases you can actually return a reference from inside a function, e.g. (below). The details on why are a bit complex, but from what I gather, the compiler creates it in the `'static` lifetime in this case. (Try it, it'll compile fine) `struct foo { val:i32 }` and `fn try_create<'a>() -> &'a foo { &foo{val:3} }` – Kobato Sep 15 '21 at 17:08
  • 1
    @JohnDoe is that not covered in [the question linked in the answer](https://stackoverflow.com/q/50345139/155423)? – Shepmaster Sep 15 '21 at 17:10
  • @Shepmaster : Certainly ;) I was mostly just putting that there more as a btw to anyone else reading it in case they missed it (or for Google searches - actually mostly cause I came from a Google search too haha). – Kobato Sep 15 '21 at 17:21
  • Btw, if your closures are returning `Option<&T>` enums, and you want to `.clone()` the data so it can be `moved` to the next call/closure, use `.cloned()` instead! The latter makes a clone of the value inside the Option enum, versus the former, which simply clones the "wrapper" but keeps holding the contents by-reference (causing the error to remain). – Venryx Jan 30 '22 at 12:29
  • I can't thank enough for this solution. I was stuck due to similar issue for 3 days. This tip solved it. :) – Vaibhav Kamble Apr 07 '22 at 13:30
  • "*In rare cases, you can use unsafe code to return the owned value and a reference to it.*"—can you? Surely by returning the owned value, it is being moved—and therefore it is UB for there also to be a reference that borrows from it? – eggyal May 20 '22 at 18:00
  • @eggyal true, it's a bit of lax wording on my part. You aren't really returning a reference to the owned value itself, but to something "within" it. That something has to be guaranteed to not move when the outer value moves. a `Box` is the canonical example. Moving the `Box` itself doesn't move the value located on the heap. – Shepmaster Jun 21 '22 at 16:45
  • But is it not UB to move (ownership of) a `Box` whilst there exists a `&T` that borrows from its heap alloc? – eggyal Jun 21 '22 at 19:29
  • @eggyal to my knowledge, no. Moving the `Box` doesn't change the address of the `T`, so the `&T` should continue to be valid. – Shepmaster Jun 21 '22 at 19:32
  • Sure, I understand the `T` shouldn't relocate... I just thought that moving ownership of the owning entity would be a violation Rust's ownership model giving rise to UB. I don't know this for sure though. – eggyal Jun 21 '22 at 19:34
  • To put it another way, what lifetime could the reference have if the owner is able to be moved whilst that borrow remains alive? – eggyal Jun 21 '22 at 19:42
  • @eggyal that's not expressible using safe Rust, thus part of the problem. Lifetimes indicate "how long is this value valid at this specific memory address?". However, Rust doesn't have a way to model the fact that the `Box` wrapper and the heap-allocated value are related but not precisely one-to-one. That's why `unsafe` is needed — to say "it's OK compiler, I'm going to guarantee it this time". – Shepmaster Jun 21 '22 at 19:47
23

Is there any way to return a reference from a function without arguments?

No (except references to static values, but those aren't helpful here).

However, you might want to look at OpenOptions::create. If you change your first line in main to

let  f = OpenOptions::new().write(true).create(true).open(b"foo.txt");

the file will be created if it does not yet exist, which should solve your original problem.

fjh
  • 12,121
  • 4
  • 46
  • 46
20

You can not return a reference pointing to a local variable. You have two alternatives, either return the value or use a static variable.

Here is why:

References are pointers to memory locations. Once functions are executed, local variables are popped off the execution stack and resources are de-allocated. After that point, any reference to a local variable will be pointing to some useless data. Since it is de-allocated, it is not in our program's possession any more and OS may have already given it to another process and our data may have been overwritten.

For the following example, x is created when the function runs and dropped off when the function completes executing. It is local to the function and lives on this particular function's stack. Function's stack holds local variables.

When run is pop off the execution stack, any reference to x, &x, will be pointing to some garbage data. That is what people call a dangling pointer. The Rust compiler does not allow to use dangling pointers since it is not safe.

fn run() -> &u32 {
    let x: u32 = 42;

    return &x;
} // x is dropped here

fn main() {
    let x = run();
}

So, that is why we can not return a reference to a local variable. We have two options: either return the value or use a static variable.

Returning the value is the best option here. By returning the value, you will be passing the result of your calculation to the caller, in Rust's terms x will be owned by the caller. In our case it is main. So, no problem there.

Since a static variable lives as long as the process runs, its references will be pointing to the same memory location both inside and outside the function. No problem there either.

Note: @navigaid advises using a box, but it does not make sense because you are moving readily available data to heap by boxing it and then returning it. It does not solve the problem, you are still returning the local variable to the caller but using a pointer when accessing it. It adds an unnecessary indirection due to de-referencing hence incurring additional cost. Basically you will be using & for the sake of using it, nothing more.

snnsnn
  • 10,486
  • 4
  • 39
  • 44
  • 10
    `return` at the end of a block like that is not idiomatic. – Shepmaster Sep 11 '19 at 18:36
  • 3
    First answer is unnecessarily verbose, second one is not elaborate enough. return is chosen for the emphasis. – snnsnn Sep 11 '19 at 18:40
  • 6
    Despite the non-idiomatic `return` I found this answer to have the clearest explanation. – jla Jan 22 '20 at 02:25
  • This makes a lot more sense. The compiler error message could maybe a bit more obvious. I had a case where I was passing in an owned object, but was returning only references to the now moved object which is going to get dropped at the end of the function. I'm more used to reading the more common "`x.foo()` borrows `x` here but `x` is dropped at the end of the function" – Josh Bowden Jan 12 '22 at 05:58
14

This is an elaboration on snnsnn's answer, which briefly explained the problem without being too specific.

Rust doesn't allow return a reference to a variable created in a function. Is there a workaround? Yes, simply put that variable in a Box then return it. Example:

fn run() -> Box<u32> {
    let x: u32 = 42;
    return Box::new(x);
} 

fn main() {
    println!("{}", run());
}

code in rust playground

As a rule of thumb, to avoid similar problems in Rust, return an owned object (Box, Vec, String, ...) instead of reference to a variable:

  • Box<T> instead of &T
  • Vec<T> instead of &[T]
  • String instead of &str

For other types, refer to The Periodic Table of Rust Types to figure out which owned object to use.

Of course, in this example you can simply return the value (T instead of &T or Box<T>)

fn run() -> u32 {
    let x: u32 = 42;
    return x;
} 
btwiuse
  • 2,585
  • 1
  • 23
  • 31
  • 4
    This answer is wrong and misguiding. Why would you box a readily available data then return it. It adds unnecessary indirection and hence cost. – snnsnn Jan 12 '22 at 06:11
  • @snnsnn the u32 variable is only for demonstration purpose. I already pointed it out at the end of the answer. – btwiuse Jan 12 '22 at 17:46
  • 2
    Even if you box a variable that is suitable for boxing, the example is wrong all the same because references are for passing a reference from outside scope into the function, in other words for borrowing the outer variable, to avoid the clutter of taking in and returning back. Your example is inverts this logic all the way around. It will be confusing for new comers and has no real use case. – snnsnn Jan 12 '22 at 20:15
  • @snnsnn Allocating value on the heap by Box makes sense if the Box's lifetime is properly long enough. In navigaid's way, Box's lifetime is too short, so it might not make sense as you say. Box should be dropped when the scope ends. So we must use Box::into_raw or Box:from_raw. See here too: https://stackoverflow.com/questions/66196972/how-to-pass-a-reference-pointer-to-a-rust-struct-to-a-c-ffi-interface – lechat Aug 05 '22 at 18:38
3

The question was:

Is there any way to return a reference to a variable created in a function?

Answer: Yes, it's possible! See the examples below for proof.

Disclaimer: It's usually better to just return a value instead, but not always...

You have to find a way to extend the lifetime. One way to do this is to create a dummy/default value outside the function, and then give this as a mutable reference (&mut T) to the function. Now the function can fill/replace the value and then return a reference (&T) to that value. You also have to specify the lifetime so that the returned reference get the lifetime of a the value created outside the function ('a).

This works because the function returns a reference to a memory location that was allocated before calling the function, so it's not deleted (or moved) when the function goes out of scope. Also the value isn't owned by the function.

Examples that proves it's possible:

//&mut T -> &T
fn example2<'a>(life: &'a mut Vec<i32>) -> &'a Vec<i32> {
    *life = vec![1, 2, 3, 4];
    life
}

fn test_example2() {
    //Could also use Vec::new()
    let mut life = Vec::default();
    let res = example2(&mut life);
    println!("{:?}", res)
}

fn test2_example2() {
    let life = &mut Vec::default();
    let res = example2(life);
    println!("{:?}", res)
}

//shows real use case
fn get_check_test_slices2<'a>(
    lifetime: &'a mut Vec<usize>,
    limit: usize,
) -> impl Iterator<Item = (&'a [usize], &'a [usize])> + 'a {
    // create a list of primes using a simple primes sieve
    *lifetime = primes1_iter_bitvec(limit).collect::<Vec<_>>();
    // iterate through pairs of sub slices without copying the primes vec
    // slices will be used to check that a complicated sieve is correct
    all_test_check_slices(lifetime)
}

Examples using a wrapper type called LateInit, with helper methods:

fn late_init_example1<'a>(holder: &'a mut LateInit<Vec<i32>>) -> &'a Vec<i32> {
    //create a new vec inside the function
    let v = vec![1, 2, 3];
    //insert it into holder and get reference to value
    let res = holder.init(v);
    //return reference
    res
}

fn late_init_example2<'a>(holder: &'a mut LateInit<Vec<i32>>) -> &'a Vec<i32> {
    //create new vec, insert it into holder, return a reference
    holder.init(vec![1, 2, 3])
}

fn using_late_init_examples() {
    let mut a = LateInit::new();
    println!("example1: {:?}", late_init_example1(&mut a));
    println!("example1: {:?}", late_init_example1(&mut LateInit::new()));
    let b = &mut LateInit::new();
    println!("example2: {:?}", late_init_example2(b));
    //can re-use the same late init
    for i in 0..=4 {
        println!("example2: {:?}", late_init_example2(b));
    }
}

/// A thin wrapper around Option<T>.
/// Enables returning a reference to a value created inside a function.
/// Note: You probably have to add lifetime annotations.
/// 1: This can be achieved by creating a
/// late initialized value on the stack first (Option<T>),
/// 2: then calling a function with a mutable reference
/// to this late initialized value,
/// 3: then initializing the value inside the function,
/// 4: and finally returning a reference to the now initialized value.
/// The returned reference can even be stored in a struct or
/// given to another function, as long as the
/// lifetime annotations are correct and
/// the late init value isn't deleted or moved.
/// TODO: better name?
/// TODO: better documentation.
pub struct LateInit<T> {
    value: Option<T>,
}

impl<T> LateInit<T> {
    /// Creates a new LateInit with None.
    /// Same as default.
    pub fn new() -> Self {
        Self { value: None }
    }

    /// Inserts the value
    /// and returns a mutable reference with 
    /// the same lifetime as the inserted value.
    pub fn init_mut<'a>(&'a mut self, init_value: T) -> &'a mut T {
        self.value.insert(init_value)
    }

    /// Inserts the value
    /// and returns a reference with 
    /// the same lifetime as the inserted value.
    /// Is non mutable init neccesary?
    pub fn init<'a>(&'a mut self, init_value: T) -> &'a T {
        self.value.insert(init_value)
    }
}

impl<T> Default for LateInit<T> {
    /// Creates a new LateInit with None.
    /// Same as new.
    fn default() -> Self {
        Self { value: None }
    }
}

Possible advantages with LateInit wrapper type (compared to just &mut T):

  • The returned reference is always to an initialized value.
  • You don't need a default constructor for T, which sometimes can be messy.
  • The intention of the code might be clearer.

TODO:
Make a crate for LateInit.
Spread the word to all similar questions that might benefit.

Edits:
&mut T examples was added (thank you Chayim Friedman).
Old wrapper type was replaced with "improved" version called LateInit.