0

In real-time applications, e.g. audio programming, you should avoid allocating memory in the heap during callbacks, because execution time is unbounded. Indeed, if your executable has run out of memory, you'll need to wait for the OS to allocate a new chunk, which can take longer than the next callback call. I could store memory on the stack, e.g. using variable-length arrays (VLA) or alloca(), but if the array is too large you get a stack overflow.

Instead, I was thinking of defining a class with an interface similar to std::vector, but that internally uses the stack if the size is smaller than a certain threshold, and the heap otherwise (I prefer a possible unbounded operation to a certain stack overflow). For the heap part I could use an std::vector or new/delete. But what about the stack? VLAs and alloca() are deallocated when they get out of scope. What alternative could I use?

I could use an std::array<T, threshold>, but that would waste memory. I expect my threshold to be in the order of 2048.

Enzo
  • 964
  • 1
  • 9
  • 20
  • Did you consider to have a `std::array` member? A drawback might be, it will always occupy that size at the stack, unless you find a way using template parameters. Providing your own allocator implementation might also be an idea. – πάντα ῥεῖ Sep 21 '18 at 16:01
  • What's the required lifetime of your allocation - is it supposed to survive the enclosing scope? What's a reasonable size threshold and how big is your stack? Is there a reason you can't just preallocate everything, or find or write a fast pool or arena allocator? – Useless Sep 21 '18 at 16:01
  • Allocating stack objects also takes time. – Jesper Juhl Sep 21 '18 at 16:02
  • It is an interesting question, but I'm unsure whether it is really on topic here, maybe a little too broad. IMHO the key here would be to use a plain `std::vector` but define a custom allocator customized for your needs. But this supposes that you can define the typical size and number of objects that you will use... – Serge Ballesta Sep 21 '18 at 16:04
  • @JesperJuhl Yes, but it isn't unbounded. There is either space on the stack or there isn't, so there is no variability. – Enzo Sep 21 '18 at 16:05
  • Also you would risk going to stack overflows quickly, if you class would be used several times in nested of recursive function calls. – πάντα ῥεῖ Sep 21 '18 at 16:06
  • @πάνταῥεῖ Edited my question. Indeed, it would waste memory. What do you mean by providing my own allocator implementation? – Enzo Sep 21 '18 at 16:06
  • @πάνταῥεῖ That is also true. – Enzo Sep 21 '18 at 16:06
  • @Enzo _"What do you mean by providing my own allocator implementation?"_ Have a look at `std::vector`s 2nd template parameter. – πάντα ῥεῖ Sep 21 '18 at 16:07
  • @Enzo The stack can grow dynamically. See for example https://llvm.org/docs/SegmentedStacks.html – Jesper Juhl Sep 21 '18 at 16:10
  • 4
    OK, I think the discussion about choosing an interface is premature until you've defined your functional requirements (eg. lifetime, size and number of elements allocated), non-functional (performance) requirements. Then see if Boost.Pool or another off-the shelf allocator is good enough. If it isn't then consider writing an [`allocator`](https://en.cppreference.com/w/cpp/memory/allocator) before writing a custom container. – Useless Sep 21 '18 at 16:11
  • Check this [thread](https://stackoverflow.com/questions/50722561/what-algorithm-to-apply-for-continuious-reallocation-of-small-memory-chunks/50722992#50722992). (+1 for boost pool if you sure you have no multi-threading when allocating memory for those vectors) – Victor Gubin Sep 21 '18 at 16:15

1 Answers1

0

If there is a possibility that data might need to be stored on the stack, and whether the size is greater than the threshold is only determined at run-time, then your type will have to include some stack container big enough to hold something of size threshold (whether std::array or something else), making your concern

I could use an std::array, but that would waste memory. I expect my threshold to be in the order of 2048.

unavoidable. I don't think there is any way around this. E.g. in code like

uint32_t N = code_that_determines_size_at_runtime();
ThresholdContainer container(N);

the compiler cannot know whether N is above or below the threshold. So for this to work, the memory layout for ThresholdContainer has to contain the stack memory for data up to size threshold, which would be unused and wasted when N > threshold. (Stitching together the stack and heap memory with some iterator that goes between the two would be horrendous, and you probably want contiguous memory).

If on the other hand, the size versus threshold is known at compile time, you could define a class templated on the size N that essentially holds an std::array if N< threshold or a vector otherwise, and provides a common interface.

jwimberley
  • 1,696
  • 11
  • 24
  • 1
    Thanks for your answer. I am not sure I understand though. You can indeed use variable-length arrays (VLA) the size of which is determined at run-time. – Enzo Sep 22 '18 at 15:47
  • @Enzo I guess I overlooked that part of your question. I'm generally required to restrict my own development to the C++ standard, so I didn't think about this. – jwimberley Sep 23 '18 at 16:38