How can a function return a dynamic number of bytes without dynamic memory allocation?

Question

I would like a function to return a dynamic number of bytes, in the range of 1K - 60K. I cannot use dynamic memory allocation, so I can't use Vec. I also cannot return a slice to a function-local array. Rust doesn't have gcc-like dynamic stack arrays. I don't believe I can resize an array, i.e. allocate a larger fixed array, then release some of the memory at the end. What other options do I have?

Welcome to Stack Overflow! It looks like your question might be answered by the answers of [Is it possible to control the size of an array using the type parameter of a generic?](https://stackoverflow.com/q/28136739/155423) or [How can I create a stack-allocated vector-like container?](https://stackoverflow.com/q/29239586/155423). If not, please **[edit]** your question to explain the differences. Otherwise, we can mark this question as already answered. — Shepmaster, Jan 21 '20 at 15:19
*cannot return a slice to a function-local array* — correct, but you can take a slice and return a subslice of it: `fn foo(data: &mut [u8]) -> &[u8]`. Then declare the array with the maximum size and pass it in. — Shepmaster, Jan 21 '20 at 15:22
If you mean a dynamic-sized array, nope, that's not (yet) possible. See https://github.com/rust-lang/rust/issues/48055 — Boiethios, Jan 21 '20 at 16:08
*Rust doesn't have gcc-like dynamic stack arrays.* You can't return dynamically sized arrays in C, either - even in the GNU dialect. — trent, Jan 21 '20 at 16:13
"I cannot use dynamic memory allocation" – I'm curious why. Are you on an embedded platform without the standard library? — Sven Marnach, Jan 21 '20 at 16:41
@Shepmaster Both the duplicates you are suggesting appear to be about arrays with sizes chosen at compile time. My understanding is that this question is about dynamically choosing the size at run time. (And I believe your suggestion to return a subslice of a slice received as parameter is the best answer.) — Sven Marnach, Jan 21 '20 at 16:49
Here's a fixed-capacity, variable-length array data structure that doesn't use heap allocation: https://docs.rs/tinyvec/0.3/tinyvec/struct.ArrayVec.html — Aloso, Jan 21 '20 at 19:12
@SvenMarnach I've done profiling of similar code where allocation turned out to be the primary overhead. I'm also planning for this code to be used embedded eventually, and so need to avoid allocation regardless of performance. — puritii, Jan 21 '20 at 20:56
`tinyvec` should handle any size that would fit on the stack with a static-sized array. However, even with `tinyvec` you're stuck with overallocating and shrinking if you want an exact-sized answer. I'd still recommend using it, because otherwise you're just going to end up using unsafe to use `realloc` or `memmove` or something. The only other real solution is to arena allocate your arrays (i.e. prealloc a huge pool on the stack, fill the first index, then advance the memory pointer to the end, but then you're just doing manual memory management). — Linear, Jan 21 '20 at 22:08

score 1 · Answer 1 · answered Jan 23 '20 at 22:57

It looks like the thing you are looking for is not possible.

I cannot use dynamic memory allocation

For me it sounds like compiler-generated code can't use dynamic memory allocation (heap) as well. Then the only place where you can save the whole result is caller's stack. The stack size have to be known at compile time.

The possible solution may be to return array with predefined size and the size of actual data. Like

fn generate_data() -> ([i32; 50000], usize)

Or you may allocate memory on caller's stack and then provide reference to data-generation function:

fn generate_data(out: &mut [i32; 50000]) -> usize

Otherwise if you are OK with heap allocation by compiler-generated code then it's not clear why you can't use dynamic allocations by yourself.

How can a function return a dynamic number of bytes without dynamic memory allocation?

1 Answers1