3

Python has a maximum size for lists, and according to the documentation, calling 

import sys
print(sys.maxsize)

will return the maximum size. On my machine, calling this code returns 9223372036854775807 = 2^63 - 1.

On the other hand, calling something like 

a = [1]*(2**32)

returns a memory error. 2^32 is much smaller than 2^63-1 - is there any way to initialize a list of 2^32 items while avoiding a memory error?

KReiser
  • 153
  • 6
  • 2
    Can you give a reason why you would want a list that large? – Tomalak Jul 12 '18 at 05:04
  • My computer does not give a memory error, it just takes forever – whackamadoodle3000 Jul 12 '18 at 05:05
  • @Tomalak Computing lots of values of an arithmetic function relatively quickly via sieving. – KReiser Jul 12 '18 at 05:07
  • 2
    The [`MemoryError`](https://docs.python.org/3/library/exceptions.html#MemoryError) indicates that your system ran out of memory while trying to create that list; also the size of the list does not have a 1:1 correlation to actual bytes consumed (see: [Size of list in memory](https://stackoverflow.com/questions/7247298/size-of-list-in-memory)). How much RAM is available for the Python process? – metatoaster Jul 12 '18 at 05:11
  • @metatoaster I'm not sure- how do I find that out? I have 16gb installed, for whatever that is worth... – KReiser Jul 12 '18 at 05:13
  • 2
    Do realize that an integer in Python takes 8 bytes (on a 64-bit platform), so the total memory required by that single statement takes at the minimum 34359738368 bytes (~34GB, 2 ** 32 * 8), plus the relatively minute overhead. – metatoaster Jul 12 '18 at 05:20
  • @metatoaster presumably my machine can handle this via virtual memory (perhaps slowly, but it should be able to do it). Why should the computer fail to do this? – KReiser Jul 12 '18 at 05:54
  • 1
    Without you conveying to the rest of us the exact limits of your computer, the current memory contention by other process on it, your operating system and its virtual memory configurations, we won't be able to answer your question. – metatoaster Jul 12 '18 at 06:02
  • 1
    If the question is rather how to have that big of a data set that contains only integers, you can always use array.array which would be slower but more memory efficient. – Işık Kaplan Jul 12 '18 at 06:13

0 Answers0