5

For example: 

InitEmployee()
{
    vector<Employee> employeeList = {
        Employee("Clark Kent",0),
        Employee("Bruce Wayne",1),
        ...
        Employee("Hal Jordan",65535)
    }
}

I cannot query from file or DB as this program needs to be in a single executable so all the constant data must be hard coded. I'm actually using boost's multi_index_container for fast lookup by name and id but for simplicity sake I used vector here as example. The problem is that I cannot have that many (2^16) constant data in a single function without stack overflow. Are there any better ways initialize this list without splitting up the function?

I'm using VC12. Thanks!

Update

See chosen answer. As others have mentioned using static will force it to go on data rather than stack. This is what I ended up with:

InitEmployee()
{
    static Employee employeeList[] = {
        {"Clark Kent",0},
        {"Bruce Wayne",1},
        ...
        {"Hal Jordan",65535}
    }

    vector<Employee*> employeeVec;
    int count = sizeof(employeeList) / sizeof(Employee);
    for (int i = 0; i < count; i++)
    {
        employeeVec.emplace(&employeeList[i]);
    }
}

The thing was Employee class used a string class rather than c-string, so I didn't want two copies of it in memory. This way I end up with just the extra memory for pointers, which is still a lot but I believe this is the best option! Also works with multi_index_container! Thanks

Community
  • 1
  • 1
Nah
  • 331
  • 2
  • 9
  • 1
    You can make the stack size bigger with the `/STACK` linker switch. Or you can declare the variable `static`, which will cause it to go in the data segment instead of the stack. But... *Why?* – j_random_hacker Apr 01 '14 at 20:34
  • 2
    Compilers don't like large amounts of data - why not stick it in a file and just read it in. – cup Apr 01 '14 at 20:37
  • If this is a Windows program, then stick the data in the resources and not in the program code. There is an RCDATA resource that is reserved for "hard-coded". – PaulMcKenzie Apr 01 '14 at 20:46
  • For large array, char* array would be a good solution. You can write a wrapper class to return Employee when needed. – camino Apr 01 '14 at 20:46
  • I think you must have misunderstood my comment, but your code still has two copies of the string data. Like I said, there is no way to get around it that, unless your `Employee` class simply stores char pointers, rather than string objects. – Benjamin Lindley Apr 01 '14 at 22:24
  • In fact, the way you have it now, you are storing even *more* redundant data. In addition to the 65536 string literals, and the 65536 pointers that point to them, you are also storing 65536 additional integers (as part of the initialization list). – Benjamin Lindley Apr 01 '14 at 22:31
  • Oohh I see I just reread your comment. Well the thing is the employee id actually non consecutive so I can't initialize it using loop index. I put it as consecutive numbers in the example for simplicity sake. In this case I have to have redundant integers anyways I believe? – Nah Apr 01 '14 at 22:46
  • Yes, if the numbers are not consecutive, and you cannot find some pattern in them that can be derived from a loop index, then you will have to redundantly store them. – Benjamin Lindley Apr 01 '14 at 22:49

4 Answers4

5

Use a static array to store the initialization data.

InitEmployee()
{
    static char const* const names[] = {
        "Clark Kent",
        "Bruce Wayne",
        ...
        "Hal Jordan"
    };

    size_t const n = sizeof(names) / sizeof(*names);

    vector<Employee> employeeList;
    employeeList.reserve(n);
    for (size_t i=0; i<n; ++i)
        employeeList.emplace_back(names[i],i);
    ...
}
Benjamin Lindley
  • 101,917
  • 9
  • 204
  • 274
  • Hi! I think this may actually work for me but I'm curious doesn't this mean the program will have 2 copies of the data? One from the char[] array and second in the container? Thanks! – Nah Apr 01 '14 at 20:54
  • The program will have to have two copies of the data, unless your employee class doesn't actually store strings, but only pointers to char arrays. The string literals have to be stored somewhere. In other words, in your original example, when you call the constructor like this: `Employee("Clark Kent",0)`, the string literal `"Clark Kent"` needs to be stored in the program's data. It is then copied to the string in your `Employee` object. – Benjamin Lindley Apr 01 '14 at 20:56
  • @Nah: However, this does come at the cost of storing 65536 extra pointers, which will be either 1/4 or 1/2 of a MiB, depending upon your platform. – Benjamin Lindley Apr 01 '14 at 21:02
  • Make it `static char const *const names[]`, then the compiler will be able to place the whole thing in a read-only data area of the executable. Without the extra `const`, it cannot do this because you might write `names[0] = "foo";` etc. – M.M Apr 01 '14 at 21:03
  • Even if it is just `char const *const names[]` the compiler SHOULD put it in read-only data , rather than allocating some stack. Alternatively , if you declare it at file scope then there is no chance of that happening anyway. – M.M Apr 01 '14 at 21:06
0

Are you sure that too many string literals would cause stack overflow? Even if they would, you can try to put them in global scope:

const char *data = R"(
Clark Kent,0
Bruce Wayne,1
...
Hal Jordan,65535)"

/* ... */

int main()
{
    /* ... */
}

If it will compile fine with huge string literal, you can even try to use raw data from some DB file instead of list.

HolyBlackCat
  • 78,603
  • 9
  • 131
  • 207
0

Since you're saying you're using VC 12, and you have one executable (and no data file) then place the hard-coded data within the resources, not the program code.

The resource that fits would be the RCDATA type. http://msdn.microsoft.com/en-us/library/windows/desktop/aa381039(v=vs.85).aspx

PaulMcKenzie
  • 34,698
  • 4
  • 24
  • 45
0

There are two options that spring to mind. The first depends on the version of Visual Studio. Are you using Visual Studio 2012, or version 12.0 (confusingly, VS2012 is actually 11.0). If you're using 2013 (ie 12.0) then you have the option of using the initializer lists which are new in C++11 (eg see the FAQ)

vector<Employee> employeeList = {
    {"Clark Kent",0},
    {"Bruce Wayne",1},
    ...
    {"Hal Jordan",65535}
};

Initializer lists rely on the C++11 rvalue move semantics to ensure that you only end up with one copy of the data in the application.

Another option could be to use boost::assign (see the docs) which means that you can use the same kind of syntax but do the allocation dynamically: 

#include <boost/assign/std/vector.hpp> // for 'operator+=()'
using namespace boost::assign; // bring operator+= into scope
vector<Employee> employeeList;
employeeList += Employee("Clark Kent", 0),
                Employee("Bruce Wayne", 1), ...
the_mandrill
  • 29,792
  • 6
  • 64
  • 93
  • I've tried this method before and it didn't work. I just assumed that that the initialized list is created on the stack anyways so it overflows – Nah Apr 01 '14 at 21:24
  • How is Employee defined? If you're using a `char*` then try using a `std::string` as that is likely to assign data on the heap if the strings are longer than (I think) 8 characters. – the_mandrill Apr 01 '14 at 21:34
  • @the_mandrill: The reason it is overflowing is because you have so much data in the instructions which insert data into the vector. In the first example, you have an initialization list with 65536 elements on the stack. It will go away at the end of the initialization, but if it causes a stack overflow, that's too late. In the second case, you have 65535 calls to the overloaded comma operator, with an Employee object as an argument to each. Each of those calls and objects is on the stack. – Benjamin Lindley Apr 01 '14 at 22:11
  • Notice in my answer, I also create an array of 65536 elements (char pointers), but it is not on the stack. Each pointer is then used in a loop to insert Employee objects into the vector. So I don't have 65536 separate function calls, but just one, which runs 65536 times. – Benjamin Lindley Apr 01 '14 at 22:14