g++: compiling a huge static std::vector

Question

I have generated a sorted list of about 117000 English words, which I have put into a std::vector<std::string> in a C++ header file, like so:

#ifndef WORDS_HPP
#define WORDS_HPP

#include <vector>
#include <string>

const std::vector<std::string> words{
    "a",
    "a\'s",
    "aa",
    "aa\'s",
    "aaa",

/* >100k lines omitted */

    "zyuganov",
    "zyuganov\'s",
    "zzz"
};

#endif

When including this monstrosity in a .cpp file and compiling the thing with g++ (Debian 4.9.2-10) 4.9.2, the compiler causes my system to run out of memory. I only have 4 GB, unfortunately.

I would like to have the vector compiled statically into my executable (toy project, FYI), but obviously I can't get it to work. How do I solve this?

@NathanOliver I would like to avoid external resources and just have the list compiled into the executable. — DiscobarMolokai, May 11 '16 at 17:24
Prefer not to place variables in header files. Every source file that includes the header file will get a copy of the global variable. This may violate the One Definition Rule (linker will complain about multiple copies). Move to a source file. — Thomas Matthews, May 11 '16 at 17:33
I currently placed it in a source file, and already decided to use a static char * with a function to build the vector of strings at runtime. I am wondering whether there is a benefit to the initializer list, though, if I can get it to compile. — DiscobarMolokai, May 11 '16 at 17:41

score 1 · Accepted Answer · edited May 23 '17 at 11:59

As R Shau says, this is a good solution:

char const* words [] = { ... };

It seems that when you initialize a std::string the compiler has to generate a call to the std::string constructor so the memory can be allocated on the heap. It doesn't seem to generate any kind of loop ... you get a lot of assembly language out the back.

It does a lot better initializing a vector of integers, but still compared to the old C style array it's a lot of overhead.

There are also various tricks for embedding arbitrary data into your C/C++ program. Some references:

Exactly what I needed +1. I currently use C style arrays and it works fine. — DiscobarMolokai, May 11 '16 at 19:37

g++: compiling a huge static std::vector

1 Answers1