0

During a lengthy computation, I need to look up some data in a number of different files. I cannot know beforehand how many or which files exactly, but chances are high that each file is used many times (on the order of 100 million times).

In the first version, I opened the file (whose name is an intermediate result of the computation) each time for lookup.

In the second version, I have a HashMap<String, Box<File>> where I remember already open files and open new ones lazily on demand.

I couldn't manage to handle the mutable stuff that arises from the need to have Files to be mutable. I got something working, but it looks overly silly:

let path = format!("egtb/{}.egtb", self.signature());
let hentry = hash.get_mut(&self.signature());
let mut file = match hentry {
    Some(f) => f,
    None => {
        let rfile = File::open(&path);
        let wtf = Box::new(match rfile {
            Err(ioe) => return Err(format!("could not open EGTB file {} ({})", path, ioe)),
            Ok(opened) => opened,
        });
        hash.insert(self.signature(), wtf);
        // the following won't work
        // wtf
        // &wtf
        // &mut wtf
        // So I came up with the following, but it doesn't feel right, does it?
        hash.get_mut(&self.signature()).unwrap()
    }
};

Is there a canonical way to get a mut File from File::open() or File::create()? In the manuals, this is always done with:

let mut file = File:open("foo.txt")?;

This means my function would have to return Result<_, io::Error> and I can't have that.

The problem seems to be that with the hash-lookup Some(f) gives me a &mut File but the Ok(f) from File::open gives me just a File, and I don't know how to make a mutable reference from that, so that the match arm's types match. I have no clear idea why the version as above at least compiles, but I'd very much like to learn how to do that without getting the File from the HashMap again.

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
Ingo
  • 36,037
  • 5
  • 53
  • 100
  • *I can't have [returning a `Result`]* — why not? You are apparently already returning a `Result`... How do you propose to handle errors otherwise? You could [`.expect("...")` the `Result`](https://doc.rust-lang.org/book/ch09-02-recoverable-errors-with-result.html#shortcuts-for-panic-on-error-unwrap-and-expect). – Shepmaster Dec 15 '21 at 16:56
  • *`HashMap>`* — why `Box` the `File`? I don't see what benefit that brings you. – Shepmaster Dec 15 '21 at 16:59
  • It looks like your question might be answered by the answers of [How can I get a reference to the key and value immediately after inserting into a `HashMap`?](https://stackoverflow.com/q/43681409/155423); [Adding entries to a HashMap and getting references to them in a for loop](https://stackoverflow.com/q/48395370/155423); [How to lookup from and insert into a HashMap efficiently?](https://stackoverflow.com/q/28512394/155423). If not, please **[edit]** your question to explain the differences. Otherwise, we can mark this question as already answered. – Shepmaster Dec 15 '21 at 17:01
  • It's hard to answer your question because it doesn't include a [MRE]. We can't tell what crates (and their versions), types, traits, fields, etc. are present in the code. It would make it easier for us to help you if you try to reproduce your error on the [Rust Playground](https://play.rust-lang.org) if possible, otherwise in a brand new Cargo project, then [edit] your question to include the additional info. There are [Rust-specific MRE tips](//stackoverflow.com/tags/rust/info) you can use to reduce your original code for posting here. Thanks! – Shepmaster Dec 15 '21 at 17:01
  • It sounds like you're creating a cache. Have you googled for how to do that in Rust? There are unique strategies for implementing caches. They can be thought of as physically mutable under the covers but logically immutable to the outside world. You may want to have an immutable API with *interior mutability*. There are numerous articles and crates that will help you with this so you don't have to reinvent the wheel. – John Kugelman Dec 15 '21 at 20:41

1 Answers1

2

Attempts to use wtf after it has been inserted into the hashmap fail to compile because the value was moved into the hashmap. Instead, you need to obtain the reference into the value stored in the hashmap. To do so without a second lookup, you can use the entry API:

let path = format!("egtb/{}.egtb", self.signature());
let mut file = match hash.entry(self.signature()) {
    Entry::Occupied(e) => e.into_mut(),
    Entry::Vacant(e) => {
        let rfile = File::open(&path)
            .map_err(|_| format!("could not open EGTB file {} ({})", path, ioe))?;
        e.insert(Box::new(rfile))
    }
};
// `file` is `&mut File`, use it as needed

Note that map_err() allows you to use ? even when your function returns a Result not immediately compatible with the one you have.

Also note that there is no reason to box the File, a HashMap<String, File> would work just as nicely.

user4815162342
  • 141,790
  • 18
  • 296
  • 355