1

With the following code that leads to a memory overflow:

import numpy as np

a = np.arange(10000000)
np.save('a', a)

l = []
while True:
    l.append(np.load('a.npy'))

Python errors out a Segmentation fault, instead of a MemoryError. It seems to be the case for machines like AWS EC2 which don't have swap memory (I've tried with a machine with more RAM and a swap memory and I get a MemoryError).

David Brochart
  • 825
  • 9
  • 17

1 Answers1

1

I am giving you multiple resources to go through below, but generally this is very much OS/system dependent. SegFlts

Simple answer: You are probably getting it because you are trying to access memory address which is not present in the memory. Its also possible to get segFaults when your memory module has issues or because of any of the following(these should clear your doubts):

Look at : Segfault instead of MemoryError when bytearray too big

Also, why segmentation fault? Possible answer

Tracing a SegFault(If you still feel something is fishy!) : python tracing a segmentation fault

Refer: Similar question

The question asked is actually broad, as the answer to it may take pages of explanation(hence added relevant links).

Community
  • 1
  • 1
Ani Menon
  • 27,209
  • 16
  • 105
  • 126
  • It looks like the second link you provided reports a similar bug in Python3, and I verified that Python2 errors out a MemoryOverflow and not a Segfault. I'm not sure Numpy uses bytearray for its data container though. Anyways, that should be fixable. Thanks for your help. – David Brochart May 21 '16 at 20:15
  • @DavidBrochart Good to know it helped :) – Ani Menon May 21 '16 at 20:22