3

How can I modify a Vec based on information from an item within the Vec without having both immutable and mutable references to the vector?

I've tried to create a minimal example that demonstrates my specific problem. In my real code, the Builder struct is already the intermediate struct that other answers propose. Specifically, I do not think this question is answered by other questions because:

Suppose I have a list of item definitions, where items are either a String, a nested list of Items, or indicate that a new item should be added to the list of items being processed:

enum Item {
    Direct(String),
    Nested(Vec<Item>),
    New(String),
}

There is also a builder that holds a Vec<Item> list, and builds an item at the specified index:

struct Builder {
    items: Vec<Item>,
}

impl Builder {
    pub fn build_item(&mut self, item: &Item, output: &mut String) {
        match item {
            Item::Direct(v) => output.push_str(v),
            Item::Nested(v) => {
                for sub_item in v.iter() {
                    self.build_item(sub_item, output);
                }
            }
            Item::New(v) => self.items.push(Item::Direct(v.clone())),
        }
    }

    pub fn build(&mut self, idx: usize, output: &mut String) {
        let item = self.items.get(idx).unwrap();
        self.build_item(item, output);
    }
}

This doesn't compile due to the error:

error[E0502]: cannot borrow `*self` as mutable because it is also borrowed as immutable
  --> src/main.rs:26:9
   |
25 |         let item = self.items.get(idx).unwrap();
   |                    ---------- immutable borrow occurs here
26 |         self.build_item(item, output);
   |         ^^^^^----------^^^^^^^^^^^^^^
   |         |    |
   |         |    immutable borrow later used by call
   |         mutable borrow occurs here

error: aborting due to previous error

For more information about this error, try `rustc --explain E0502`.

I don't know how to have the Builder struct be able to modify its items (i.e. have a mutable reference to self.items) based on information contained in one of the items (i.e. an immutable borrow of self.items).

Here is a playground example of the code.

Using Clone

@Stargateur recommended that I try cloning the item in build(). While this does work, I've been trying to not clone items for performance reasons. UPDATE: Without adding the Vec<Item> modification feature with Item::New, I implemented the clone() method in my real code and cloned the value in the equivalent of the example build() method above. I saw a 12x decrease in performance when I do self.items.get(idx).unwrap().clone() vs self.items.get(idx).unwrap(). I will continue looking for other solutions. The problem is, I'm still relatively new to Rust and am not sure how to bend the rules/do other things even with unsafe code.

Code that does work (playground)

impl Clone for Item {
    fn clone(&self) -> Self {
        match self {
            Item::Direct(v) => Item::Direct(v.clone()),
            Item::Nested(v) => Item::Nested(v.clone()),
            Item::New(v) => Item::New(v.clone()),
        }
    }
}

and change build to clone the item first:

        let item = self.items.get(idx).unwrap().clone();
d0c_s4vage
  • 3,947
  • 6
  • 23
  • 32
  • Hmm, it's similar in some ways, but I think I would end up with the same error creating an intermediate struct - I'd still have to reference an item within `Vec` to be able to construct the intermediate struct. Funny thing is, in my real code `Builder` *is* the intermediate struct already – d0c_s4vage Jun 14 '20 at 14:23
  • Oh yeah your case also have a problem cause you can't borrow the vector as immutable and them modified it. You need either to clone item to avoid the borrow, or you need to handle the match in `build` cause compiler can't know the borrow could end when you modify your vector (https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=2b275834bc3c1c9eb95741f52e2cbf31). – Stargateur Jun 14 '20 at 14:33
  • I'll try cloning the value - clones are something that I've been trying very hard to avoid though for performance reasons. – d0c_s4vage Jun 14 '20 at 14:36
  • @Stargateur I've updated my question to include a potential solution that implements the `Clone` trait for `Item`. This does work, but I'm not sure how big of a performance hit my real project will take. If I don't come up with anything else I'll mark it as an answer (if you make it a separate answer). – d0c_s4vage Jun 14 '20 at 14:50
  • @Stargateur what did you mean by *"handle the match in build"*? Do you mean to merge the `build` and `build_item` functions? That would work for some cases - the real code I'm working on is recursive, where some `Item`s contain lists of `Item`s, each of which are built recursively by the `build_item` function. – d0c_s4vage Jun 14 '20 at 14:56
  • every recursive can be write as imperative, but, without more concrete example I can't help you more, you maybe implementing your tool with the wrong solution. Your question is good but 1. it's not clear what is your purpose here 2. you forget to show you use recursivity. – Stargateur Jun 14 '20 at 15:04
  • Good points @Stargateur! I'll update the question. I was trying to keep the question minimal without bringing in the recursive aspect. – d0c_s4vage Jun 14 '20 at 15:08
  • @Stargateur Thanks again for your help! I benchmarked my real code with `clone()` and without `clone()` (removing the ~`Item::New` functionality) and saw a 12x decrease in perf :^( I'll keep looking for solutions – d0c_s4vage Jun 14 '20 at 21:41
  • 2
    Could you give asome more context, to explain what it is that you are really trying to do? Because there might be a better way to structure the data for better performance without running into borrow-checking errors. For instance, one possibility would be using an arena instead of a `Vec`, but without more context it's hard to know if this would be suitable for you. One important thing to recognize here is that Rust is not being a nuisance here with the compiler error: it is protecting you from actual memory corruption which would happen when the Vec reallocates and leaves your &Item dangling. – Brent Kerby Jun 14 '20 at 21:51
  • Also, by the way, your implementation of `Clone` could be automatically generated by putting `#[derive(Clone)]` above the `enum Item`. – Brent Kerby Jun 14 '20 at 21:54
  • @BrentKerby This is part of a grammar fuzzer that I am working on - https://gitlab.com/d0c-s4vage/resmack. It uses defined grammars to generate random, structured data. The sample code is very similar to the actual code - a `Vec` is used to track all rules, with indices being used to access+build specific rules if referenced by another rule. The code in the link ^ works fine right now, and I am sure I am approaching things wrong in places (first Rust project). This specific question is about a new feature that I want to add, where new rule variants can be added during rule generation. – d0c_s4vage Jun 15 '20 at 00:07
  • @BrentKerby circling specifically back to this question, perhaps I can simplify it even more. I want to be able to append to a `Vec`, and have immutable access to individual items at the same time. Individual items will never be modified once in the `Vec`. – d0c_s4vage Jun 15 '20 at 00:13

1 Answers1

3

Whenever approaching problems like this (which you will encounter relatively frequently while using Rust), the main goal should be to isolate the code requiring the immutable borrow from the code requiring the mutable borrow. If the borrow from the items vec in build is unavoidable (i.e. you cannot move the item out of self.items or copy/clone it) and you must pass in a reference to this item to build_item, you might want to consider rewriting your build_item function to not mutate self. In this case, build_item only ever appends new items to the end of self.items, which lets us make an interesting refactor: Rather than have build_item modify items, make it return the items to be added to the original vector, and then have the caller add the newly generated items to the items vector.

impl Builder {
    fn generate_items(&self, item: &Item, output: &mut String) -> Vec<Item> {
        match item {
            Item::Direct(v) => {
                output.push_str(v);
                Vec::new()
            }
            Item::Nested(v) => {
                v.iter()
                    .flat_map(|sub_item| self.generate_items(sub_item, output))
                    .collect()
            }
            Item::New(v) => vec![Item::Direct(v.clone())],
        }
    }

    pub fn build_item(&mut self, item: &Item, output: &mut String) {
        let mut new_items = self.generate_items(item, output);
        self.items.append(&mut new_items);
    }

    pub fn build(&mut self, idx: usize, output: &mut String) {
        // Non lexical lifetimes allow this to compile, as the compiler
        // realizes that `item` borrow can be dropped before the mutable borrow

        // Immutable borrow of self starts here
        let item = self.items.get(idx).unwrap();
        let mut new_items = self.generate_items(item, output);
        // Immutable borrow of self ends here

        // Mutable borrow of self starts here
        self.items.append(&mut new_items);
    }
}

Note that in order to preserve the API, your build_item function has been renamed to generate_items, and a new build_item function that uses generate_items has been created.

If you look carefully, you'll notice that generate_items doesn't even require self, and can be a free-standing function or a static function in Builder.

Playground

EvilTak
  • 7,091
  • 27
  • 36
  • 1
    This makes sense, and makes me consider the exact use cases/limitations that I am comfortable with. Adding the new items only after `generate_items()` has been called would mean that the new items are unavailable until `generate_items()` returns. The pros/cons of this for my real code is beyond the scope of this question though - thank you for the answer/help! Learning Rust has been an adventure, to say the least :^) – d0c_s4vage Jun 15 '20 at 00:18