1

I want to create an empty 3D matrix of dimensions (1024,1024,360).
When I do np.zeros((1024,1024,360)), I get the following error :

ValueError: array is too big.

What should I do?

mpromonet
  • 11,326
  • 43
  • 62
  • 91
Prakriti Kumari
  • 49
  • 1
  • 12
  • 4
    You are trying to store approximately 360 million numbers. Not all computers can handle that amount of information...! Try and think of an alternative way to approach the problem. It will save you significant computation time. – gtlambert Sep 09 '15 at 17:37
  • 2
    possible duplicate of [What's the maximum size of a numpy array?](http://stackoverflow.com/questions/14525344/whats-the-maximum-size-of-a-numpy-array) – Daniel Sep 09 '15 at 17:39
  • Then is there any alternative way to store them? – Prakriti Kumari Sep 09 '15 at 17:41
  • @PrakritiKumari To store them for what? You can easily store that amount of information on a hard disk or even a jump drive, but you're going to have a hard time performing computations on them that way. You have to spell out your intentions. – Two-Bit Alchemist Sep 09 '15 at 17:42
  • I am performing 3D reconstruction for which I need to need to create an empty matrix and then store the reconstructed 3D image in it. – Prakriti Kumari Sep 09 '15 at 17:44

1 Answers1

0

np.zeros, as you're using it, will return a float array, where each element is 8 bytes. You are trying to store 1024 x 1024 x 360 x 8 bytes, which is roughly 3 gigabytes according to Google.

Do you have 3GB available? Or rather, do you need floats, or could you get by with another dtype (e.g. uint8 for image data, which instead lands you at ~0.38 GB).

Edit:

If memory isn't your strongpoint, consider a memory mapped array: http://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html

trianta2
  • 3,952
  • 5
  • 36
  • 52