C++ How to store large binary lookup table with application?

Question

I suspect what I want to do is create something like a binary library or .obj that I can just feed the linker, but I'm not even sure which of TFM to R for this.

I have a large binary LUT (2 million 32 bit values.) It takes a long time to calculate and I really want to avoid recomputing it each time the application runs. I'd also prefer not to store it as a separate file and read it in, which is what I am doing now.

For smaller LUTs I'd normally just do a header file with suitable entry declarations, but in this case that seems fairly untenable and a will generate lot of overhead just generating the header file.

Development target and environment is a CMAKE application created by Visual Studio 2019 Community Edition.

score 0 · Answer 1 · answered Feb 27 '20 at 22:10

0

If you are using C++14 or later you can probably use constexpr to make your LUT at compile time.

This maybe kinda what you are looking for? https://stackoverflow.com/a/37413361/12743421

answered Feb 27 '20 at 22:10

GrendleM

350
2
9

I think that may be exactly what I need. I'll give it a try, thanks! – ozindfw Feb 28 '20 at 16:32
Took a while to get back to this, I can't seem to make it work because my LUT function includes sin() as part of the formula for value initialization, and that's not a constant function. I'm still filling the LUT for each run. – ozindfw Apr 21 '20 at 20:18

score 0 · Answer 2 · edited Jan 31 '22 at 20:10

I did a similar task in the following way:

In the header file say it MySpecialArray.h there is just external array declaration:

enum {
    uMySpecialArraySize = 1000000,
};
extern const unsigned* g_pMySpecialArray;

Generate (with a special run of definable block or external project) a file MySpecialArray.cpp with the actual LUT data as a static C-array:

#include "stdafx.h"

#include "MySpecialArray.h"

const unsigned g_MySpecialArray[uMySpecialArraySize] = {
    0x12345678, 0x12345678, 0x12345678, 0x12345678, 0x12345678, 0x12345678, 
    ...
};

const unsigned* g_pMySpecialArray = g_MySpecialArray;

This worked for me for many years. Array is easily accessible from the code and the data block is inside compiled exe. Compile time of mid-size project is normal and there was no significant change with this addition as far as I remember.

My LUT is a few times smaller than requested millions but hope it should work on millions as well. The question is a bit old, but maybe someone else will seek for solution.

C++ How to store large binary lookup table with application?

2 Answers2