0

I am coding some c++ lines to create a float array t[679][679][432][586] to store float values, I compile using g++ -mcmodel=medium my_code.c -o my_code.out and that compiles well, but when executing with ./my_code.out 'option1' 'option2' I get the error segmentation fault, I think its related to lot of memory needed to be allocated for this long array (I have 8 Gb of ram), so if it is the case: please help me with the best solution to optimize the amount of memory needed or if 16 Gb of ram will help I can extend it. If there is an other potential origin of this error please let me know.

Martin

Martin Sower
  • 1
  • 5
    Some quick math shows that this array will take 466 GB of memory... – 0x5453 Dec 16 '20 at 16:22
  • 1
    You're going to need a different approach because that's not going to happen, at least not on a machine with 8GB of memory, and *especially* not on the stack. As 0x5453 points out, this takes up `sizeof(float) * 679 * 679 * 432 * 586` bytes, or 434GB of memory. – tadman Dec 16 '20 at 16:25
  • Why do you need an array this big, *especially* one with that many dimensions? You're probably going to need to re-think how you're tackling this problem unless you can spring for a machine with that much memory. Can you cram 512GB of memory in your machine? Could you do it some other way, like using a file instead, or a sparse array? – tadman Dec 16 '20 at 16:28
  • If you give us more information about what you are actually trying to do, we might be able to hint you at a better solution. – Lukas-T Dec 16 '20 at 16:40
  • Thank you @tadman @churill for the informations, so to explain, I am browsing a file that contains list of events (physics events) to sort them, what I need is to store the number of events that occured between the `a ` and the `b` detectors (for all combinations of `a` and `b`), and for each pair `a, b` sort the events according to their angle `c` and axial location `d`. Then `t[a][b][c][d] ` will contain the number of events occured between detectors `a` and `b` and having angle `c` and location `d`, with `679, 679, 432, 586` the numbers of possible values of `a, b, c, and d` respectiv ` . – Martin Sower Dec 16 '20 at 18:55
  • 1
    Unless every possible interaction *will* happen, what you need is a "sparse array", that is the whole array doesn't actually exist, but it is filled out as events happen. One cheap way to do this is to use a `std::map` where you compute a 64-bit "vector" based on those four values. – tadman Dec 17 '20 at 02:56
  • Thank you @tadman for your help, I will use this solution, I want to add that I am storing a count (number of events) so, can I use int (sparse array of ints) to save more memory? and how to do it? – Martin Sower Dec 17 '20 at 20:59
  • Use the smallest possible representation. If the number of events is under 65535, consider using `uint16_t`. – tadman Dec 18 '20 at 01:46

0 Answers0