How to create a pool of contigous memory across multiple structs in Rust?

Question

I'm trying to create a set of structs in Rust that use a contiguous block of memory. E.g:

<------------ Memory Pool -------------->
[  Child  |  Child  |  Child  |  Child  ]

These structs:

may each contain slices of the pool of different sizes
should allow access to their slice of the pool without any blocking operations once initialized (I intend to access them on an audio thread).

I'm very new to Rust but I'm well versed in C++ so the main hurdle thus far has been working with the ownership semantics - I'm guessing there's a trivial way to achieve this (without using unsafe) but the solution isn't clear to me. I've written a small (broken) example of what I'm trying to do:

pub struct Child<'a> {
    pub slice: &'a mut [f32],
}

impl Child<'_> {
    pub fn new<'a>(s: &mut [f32]) -> Child {
        Child {
            slice: s,
        }
    }
}

pub struct Parent<'a> {
    memory_pool: Vec<f32>,
    children: Vec<Child<'a>>,
}

impl Parent<'_> {
    pub fn new<'a>() -> Parent<'a> {
        const SIZE: usize = 100;
        let p = vec![0f32; SIZE];
        let mut p = Parent {
            memory_pool: p,
            children: Vec::new(),
        };
        // Two children using different parts of the memory pool:
        let (lower_pool, upper_pool) = p.memory_pool.split_at_mut(SIZE / 2);
        p.children = vec!{ Child::new(lower_pool), Child::new(upper_pool) };
        return p; // ERROR - p.memory_pool is borrowed 2 lines earlier
    }
}

I would prefer a solution that doesn't involve unsafe but I'm not entirely opposed to using it. Any suggestions would be very much appreciated, as would any corrections on how I'm (mis?)using Rust in my example.

Hi there! I think your question is answered by [this answer](https://stackoverflow.com/a/32300133/2408867). I didn't dupe-close your question immediately though, because maybe you can work around this limitation. When does your vector of `f32` change? Have you considered what will happen if it reallocates? — Lukas Kalbertodt, Aug 23 '19 at 11:02
How would you have done this in C++? You can't put references in structs, and standard C++ doesn't have `restrict`/`&mut`, so `Child` doesn't translate at all. Would you have used raw pointers? That's the `unsafe` solution in Rust. Or would you have used indices? You can do that in Rust without `unsafe`. How would you have handled reallocation of the underlying `std::vector`? Whatever your solution for this is in C++, I can virtually guarantee you could do the same thing in Rust. But if you're reaching for Rust-specific tools like lifetimes and `&mut`, you won't be able to replicate it. — trent, Aug 23 '19 at 12:08

score 0 · Answer 1 · answered Aug 23 '19 at 12:12

Yes, it's currently impossible (or quite hard) in Rust to contain references into sibling data, for example, as you have here, a Vec and slices into that Vec as fields in the same struct. Depending on the architecture of your program, you might solve this by storing the original Vec at some higher-level of your code (for example, it could live on the stack in main() if you're not writing a library) and the slice references at some lower level in a way that the compiler can clearly infer it won't go out of scope before the Vec (doing it in main() after the Vec has been instantiated could work, for example).

edwardw · Answer 2 · 2019-08-23T14:02:50.847

This is the perfect use case for an arena allocator. There are quite a few. The following demonstration uses bumpalo:

//# bumpalo = "2.6.0"

use bumpalo::Bump;
use std::mem::size_of;

struct Child1(u32, u32);

struct Child2(f32, f32, f32);

fn main() {
    let arena = Bump::new();
    let c1 = arena.alloc(Child1(1, 2));
    let c2 = arena.alloc(Child2(1.0, 2.0, 3.0));
    let c3 = arena.alloc(Child1(10, 11));    

    // let's verify that they are indeed continuous in memory
    let ptr1 = c1 as *mut _ as usize;
    let ptr2 = c2 as *mut _ as usize;
    let ptr3 = c3 as *mut _ as usize;
    assert_eq!(ptr1 + size_of::<Child1>(), ptr2);
    assert_eq!(ptr1 + size_of::<Child1>() + size_of::<Child2>(), ptr3);
}

There are caveats, too. The main concern is of course the alignment; there may be some padding between two consecutive allocations. It is up to you to make sure that doesn't gonna happen if it is a deal breaker.

The other is allocator specific. The bumpalo arena allocator used here, for example, doesn't drop object when itself gets deallocated.

Other than that, I do believe a higher level abstraction like this will benefit your project. Otherwise, it'll just be pointer manipulating c/c++ disguised as rust.

How to create a pool of contigous memory across multiple structs in Rust?

2 Answers2