Static initialization of large map

Question

I am statically initializing a large (~20kb) std::unordered_map

const std::unordered_map<std::string, std::string> mapStringToString{
{"AAF", "ELN"}, {"ACT", "POC"}, {"AEK", "THJ"}, {"AFO", "EUP"},
{"AHB", "HYW"}, {"AIB", "GFW"}, {"AJX", "BUX"}, {"ALD", "FKP"},
{"ALX", "LWB"}, {"AMY", "NQB"}, {"AOI", "GUC"}, {"ASW", "VMH"},
{"ATQ", "SXK"}, {"AVL", "ENB"}, {"BCJ", "NSX"}, {"BEM", "QVR"},
{"BGU", "WPU"}, {"BJR", "ZCS"}, {"BJT", "ZTK"}, {"BOY", "FYU"},
...
{"XSJ", "FRR"}, {"XUD", "NUI"}, {"XVH", "QTI"}, {"XVJ", "TGG"},
{"XWK", "AZB"}, {"XYQ", "YTO"}, {"YAG", "ZQR"}, {"YAY", "UJY"},
{"YBN", "FEB"}, {"YCR", "EPQ"}, {"YHU", "UUD"}, {"YIG", "YMJ"},
{"YME", "EEZ"}, {"YNE", "EIU"}, {"YTC", "IOC"}, {"YTS", "JQM"},
{"YUH", "JPF"}, {"ZDY", "LFQ"}, {"ZFY", "YIH"}, {"ZMF", "BPK"},
{"ZPR", "TNG"}, {"ZTM", "DFJ"}, {"ZVB", "ZSV"}, {"ZXH", "IOA"},
{"ZZR", "RQG"}};

and code analysis is complaining about stack usage:

C6262   Excessive stack usage   Function uses '19920' bytes of stack:  exceeds /analyze:stacksize '16384'.. This allocation was for a compiler-generated temporary for 'struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > [249]' at line 0.  Consider moving some data to heap.     <no file>

This warning is reasonable if all the data in the table is put on the stack as part of the unordered_map constructor.

Is there a better way to do this initialization?

Instead of harcoding the strings, why not load them from some text file, i.e. `csv`? — A.S.H, Dec 28 '16 at 01:22
something like this http://stackoverflow.com/questions/10951447/load-stdmap-from-text-file is definitely doable, are there any other alternatives? I'd like to keep the map `const` if possible — jnnnnn, Dec 28 '16 at 01:30
tbh I dont think there is a better way. Loading the strings from a text file yields many advantages. for one, you dont have to recompile every time the keys'values change. Also, you can visualize your csv in Excel and check for duplicates or errors. — A.S.H, Dec 28 '16 at 01:33
In order to keep it const, or hide it, I would make a "getter" function that declares the map inside it as `static`, and returns a reference to it. The function should load it once. — A.S.H, Dec 28 '16 at 01:35
@jnnnnn If it's inside a function then there will be a different impact for stack usage — M.M, Dec 28 '16 at 03:29

score 3 · Accepted Answer · answered Dec 28 '16 at 02:16

A map of any reasonable size is probably best initialised from a file: aside from avoiding the problem with the stack size, it also tends to be easier to maintain. On the other hand, there is a chance that the file is not accessible for whatever reason and embedding the data, especially when it is essentially immutable, into the program may be favourable. Note, that there is no problem with constness of the resulting map: the map can be constructed by a sequence of iterator which could read from a file. Here is an example of this approach:

#include <fstream>
#include <iostream>
#include <iterator>
#include <string>
#include <unordered_map>

struct mystring
    : std::string {
};

std::istream&
operator>> (std::istream& in, std::pair<mystring, std::string>& p) {
    in >> p.first >> p.second;
    return in;
}

using pair     = std::pair<mystring, std::string>;
using iterator = std::istream_iterator<pair>;
using map_type = std::unordered_map<std::string, std::string>;
map_type const map(iterator(std::ifstream("map.txt") >> std::skipws), iterator());

int main()
{
    for (auto const& p: map) {
        std::cout << "'" << p.first << "'->'" << p.second << "'\n";
    }
}

The type mystring is needed to have a type for which the input operator can be overloaded: using only standard library types it would be necessary that the standard library defines an input operator but it doesn't do so.

Since we can, obviously, use a sequence specified by iterator rather than an std::initializer_list<...> an alternative storing the data inside the program is a static array with the corresponding elements which is then used as the underlying sequence to initialise map. For example:

#include <algorithm>
#include <iostream>
#include <string>
#include <unordered_map>

std::pair<std::string const, std::string> values[] = {
    { "foo1", "bar1" },
    { "foo2", "bar2" },
    { "foo3", "bar3" },
    { "foo4", "bar4" }
};
using map_type = std::unordered_map<std::string, std::string>;
map_type const map(std::begin(values), std::end(values));

int main()
{
    for (auto const& p: map) {
        std::cout << "'" << p.first << "'->'" << p.second << "'\n";
    }
}

Static initialization of large map

1 Answers1