Is there a size limit on the length of a std::array?

Question

If I try to instantiate a large two-dimensional std::array on our Centos 7.6 system, compiled with g++ 7.3.1, I get a segmentation fault:

int main()
{
    const unsigned NUM_BUFFERS = 200;
    const unsigned BUFFER_SIZE_BYTES = 1024 * 1024;

    std::array<std::array<unsigned char, BUFFER_SIZE_BYTES>, NUM_BUFFERS> buffer;
}

gives:

$ ./main
Segmentation fault (core dumped)

Is there a size limit on the length of a std::array?

I have declared this array on the stack but presumably the storage is on the free store. What is failing here?

"presumably the storage is on the free store" maybe it is time to challenge that presumption. — juanchopanza, Feb 04 '20 at 11:59
https://stackoverflow.com/questions/2780100/is-there-any-limit-on-stack-memory — BiagioF, Feb 04 '20 at 12:00
Local variables inside functions are typically stored on the stack. That's also true for arrays, both old C-style arrays as well as `std::array` objects. The stack is a limited resource; On e.g. Linux the default stack for a process is 8MiB. You attempt to create an array of 200MiB. — Some programmer dude, Feb 04 '20 at 12:01
BTW if it is on the stack, it would good to have a code sample that unambiguously shows that. — juanchopanza, Feb 04 '20 at 12:09
I thought that the std::array object would be on the stack, but the contents (data) would be in the free store. So I take it that is wrong. So I assume I should create the std::array object using new. — DavidA, Feb 04 '20 at 12:10
@DavidA No, what you seem to want is a `std::vector` instead. As in `std::vector> buffer(NUM_BUFFERS, std::vector(BUFFER_SIZE_BYTES));` — Some programmer dude, Feb 04 '20 at 12:12
@juanchopanza I've editing code snippet to show it is on the stack. — DavidA, Feb 04 '20 at 13:26

score 4 · Answer 1 · answered Feb 04 '20 at 12:14

4

You're getting a stack overflow.

std::array allocates in-place, and stack space is usually very limited (e.g. on Windows is 1MB, on Linux between 4-8 MB).

Use std::vector instead, which allocates on the heap.

answered Feb 04 '20 at 12:14

rustyx

80,671
25
200
267

Worth noting that in this case, a `std::vector` of `std::array`s may be the best approach. – Sneftel Feb 04 '20 at 12:16
1

@Sneftel Regarding vector of array: It was encountered recently on SO that libstdc++ has quality of implementation issue that the element size of vector could cause stack overflow. So, if rows are in Megabyte range, then it's safer to use vector of vectors. – eerorika Feb 04 '20 at 12:23
1

Or just a `std::unique_ptr` of the array. That could be default-initialized as well. – walnut Feb 04 '20 at 12:26
@eerorika Bleh. Thanks, news to me. – Sneftel Feb 04 '20 at 12:27

score 2 · Answer 2 · edited Jun 20 '20 at 09:12

Is there a size limit on the length of a std::array?

The limit on the size of an object is implementation defined. The standard specifies the minimum required limit as follows, but most implementation probably support much larger objects (up to available memory):

[implimits]

Size of an object ([intro.object]) [262 144].

I have declared this array on the stack but presumably the storage is on the free store

You've presumed wrong. std::array doesn't allocate anything from the free store, but has the array as a member.

What is failing here?

Stack overflow.

The size of the execution stack depends on the implementation. On desktop systems, the default size of stack is one to few megabytes. All automatic variables of every function call must fit into this space.

To avoid this limitation, allocate the array dynamically (for example with std::make_unique or std::vector).

Is there a size limit on the length of a std::array?

2 Answers2