0

I have read numerous other answers on this topic but they don't quite answer what I'm looking for. Examples:

Class members and explicit stack/heap allocation

When should a class be allocated on the stack instead of the heap

Member function memory allocation stack or heap?

C++ stack vs heap allocation

These answers cover the mechanical differences between the two (automatic vs manual memory management, variable lifetimes, etc) but I am more interested in best practices, and how to write code that can scale.

Context

I am writing a class which processes a large stream of data, say 10-100s's of GB. Let's assume that the performance bottleneck is how fast my class can process the data, e.g. the source and destination of the data are both fast.

My class works by splitting the data into chunks of size N bytes, and processing. The optimal size N for maximal throughput depends on the processing performed, which is only known at runtime. N can range from 10's of bytes up to 1000's of bytes. If I did everything in the stack, for say N = 256, the total sum of member variables in the class is <1MB.

I also tried stack allocating different sizes of arrays for a small set of different Ns, and using only one at a given time. This is so far the fastest implementation. Nevertheless, comparing implementations that use all stack vs heap, performance difference is fairly small to use the heap, so that ends up being simpler.

Questions

If I make the choice to use stack vs. heap now, how does that affect future users of my class? For example, in theory one could write a program that has 100's of instances of this class. If I used all stack, and the user put all my instances on the stack, it would blow up.

How is stack usage factored into the design of a large hierarchical system? I don't see that mentioned when I read online and books. Mostly the stack is mentioned in the context of excessive recursion, trying to outright declare 100MB array, etc.

Generally, is the author of a class (whose underlying workings are abstracted away) supposed to give the end user some information about the stack footprint? Or some direction on when/whether heap allocation is required?

Colin Weltin-Wu
  • 347
  • 1
  • 2
  • 10
  • 1
    i have the impression you are overthinking the issue. A class has no control over whether instances are stored on the stack or on the heap, if a user needs some `Foo`s they might use a `std::array`, if they need too many for the stack they can use a `std::vector`. `Foo` needs not care whether it will be placed on the stack or on the heap. That being said, asking for best practices is purely opinion-based – 463035818_is_not_an_ai May 20 '20 at 17:22
  • 1
    "How is stack usage factored into the design of a large hierarchical system?" it is not. When you design a large hierarchical system you do not think about stack. For example when you design a large factory in life you do not think if workers would use a hammer or an axe in each case . That details are too small and irrelevant to the task. – Slava May 20 '20 at 17:22
  • 1
    Note that you can also combine both. This technique is usually called _small buffer/data/object optimization_ and is implemented for example by Boost's `small_vector`. You can also allow user to control the used stack space via a template argument (possibly with some default value). – Daniel Langr May 20 '20 at 17:33
  • If you need to allocate a large chunk of data, the time you spend operating on that data will likely dwarf the time spent on allocation. You'll gain much more by finding faster ways to do the operation than you will by finding faster ways to allocate memory. – Caleb May 20 '20 at 17:50
  • Is the factory analogy really equivalent? In that example, the assumption is that the tools are essentially a fungible, unlimited resource. The stack is a globally shared resource, and it is pretty limited (at least relative to how quickly one could use it). As an aside, companies can achieve pretty impressive optimization by managing such things (e.g. Checklist Manifesto). The *small buffer optimization* is really cool, and along the lines of what I'm asking. It still seems like a macro-level control. I guess there is no global guideline for this. Thank you. – Colin Weltin-Wu May 20 '20 at 21:03
  • I understand that the class has no control of its instances. But, if my class footprint is 40 bytes, vs. 200kB, that makes a big difference for the end user. The statement "if they need too many for the stack" seems to depend on too many unknowns. Do most people write quick testcases with sizeof() and then make that assessment? – Colin Weltin-Wu May 20 '20 at 21:06

0 Answers0