run for-loop for large numbers in python

Question

 for i in range(1000000000):
 print(i)

This snippet gives memory error than How to run this loop. Any suggestions on how to run it.Thanks.

>>>MemoryError

UPDATE: Actually I was trying something like this:

arr = []

for i in xrange(1000000000):
    arr.append(i*i)
print(max(set(arr)))

I don't know how to deal with such large numbers.

Read about python generators: https://stackoverflow.com/questions/1756096/understanding-generators-in-python — cmaher, Jul 08 '17 at 15:14
@TimPeters thanks for the comment,it helps.Unfortunately ,now loop runs but this input is not ready to get appened in a list — Hemant, Jul 08 '17 at 15:18
Sorry, but I doubt anyone can guess what "this input is not ready to get appened in a list" could mean from anything you wrote in your question. — Tim Peters, Jul 08 '17 at 15:33

score 4 · Answer 1 · answered Jul 08 '17 at 15:15

4

on python2 this will give an error since range returns a list and in this case your list is too big for memory to hold, try in python3 and it will work. try xrange in python2.

answered Jul 08 '17 at 15:15

PYA

8,096
3
21
38

score 4 · Answer 2 · answered Jul 08 '17 at 15:19

4

One way of solving it, even though xrange in most cases is superior, is to use a while loop.

counter = 0
while (counter < 1000000000):
    print i
    counter = counter + 1

answered Jul 08 '17 at 15:19

klutt

30,332
17
55
95

score 3 · Answer 3 · answered Jul 08 '17 at 15:21

Using xrange instead of range should solve your problem

But what is the difference?

For the most part, they are the exact same. They both generate a list of integers. However, range returns a list object whereas xrange returns a xrange object.

Basically xrange doesn't generate a static list at runtime...instead it generates the values as you need them , which prevents memory error.

Happy coding!

Shivam Chaurasia · Answer 4 · 2017-07-08T15:40:31.533

The range usually returns an list of values which can be iterated. But, however if you are trying to generate a very large number of values then python Generators are best option as they yield a single value at a time and are being memory efficient to handle large scalable data.

def firstn(n):
    num = 0
    while num < n: 
        yield num
        num += 1

generatorObject = firstn(1000000000)

for i in generatorObject:
    print(i)

score 1 · Accepted Answer · answered Jul 08 '17 at 16:30

Depends on exactly what you're trying to do. Given that you're using xrange already, I'm going to assume that the issue is with the size of the array that you're creating. If you only want the largest element, you don't need to keep the whole array in memory.

max_ = float("-inf")
for n in xrange(10000000000):
    if n**2 > max_:
        max_ = n**2

print(max_)

Alternatively, if you do want the whole array, you could write the values to a file, then read the file back in chunks.

run for-loop for large numbers in python

5 Answers5