Python Infinite Integers

Question

Python 3 integers have unlimited precision. In practice, this is limited by a computer's memory.

Consider the followng code:

i = 12345
while True:
    i = i * 123

This will obviously fail. But what will be the result of this? The entire RAM (and the page file) filled with this one integer (except for the space occupied by other processes)?

Or is there a safeguard to catch this before it gets that far?

Depends on what the OS allows, and whether `i` started off as zero or not. — Patrick Maupin, Sep 20 '15 at 20:55
A non-zero `i`. So how far do you think this will get before a typical Windows system considers the process a runaway and throws a MemoryError? — mcu, Sep 20 '15 at 20:58
@coding4fun Read this: http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm — nbro, Sep 20 '15 at 20:59
So, interestingly, there may be some optimization on integers (exponent stored separately?) I started running your loop, and memory allocation hasn't budged. — Patrick Maupin, Sep 20 '15 at 20:59
I stopped it and looked, and it's just an efficiency issue. I suppose if you want to eat up RAM as well as CPU, you might want to shift more than one bit at a time. — Patrick Maupin, Sep 20 '15 at 21:01
Yeah, `i **= 2` visibly eats into memory; `i *= 2` not so much. — Patrick Maupin, Sep 20 '15 at 21:02
@PatrickMaupin, sys.getsizeof does climb, also top shows the process memory climbing but would take an age to hit a MemoryError — Padraic Cunningham, Sep 20 '15 at 21:03
Right, I saw that. The DOS would be noticed from a CPU perspective long before memory was an issue on most systems. — Patrick Maupin, Sep 20 '15 at 21:03
article about the windows process limits http://blogs.technet.com/b/markrussinovich/archive/2008/07/21/3092070.aspx — Padraic Cunningham, Sep 20 '15 at 21:16
@PatrickMaupin Could you please clarify what you mean by *DOS*? — mcu, Sep 20 '15 at 21:32

score 1 · Accepted Answer · edited May 23 '17 at 10:27

You could check what happens without risking to fill all available memory. You could set the memory limit explicitly:

#!/usr/bin/env python
import contextlib
import resource

@contextlib.contextmanager
def limit(limit, type=resource.RLIMIT_AS):
    soft_limit, hard_limit = resource.getrlimit(type)
    resource.setrlimit(type, (limit, hard_limit)) # set soft limit
    try:
        yield
    finally:
        resource.setrlimit(type, (soft_limit, hard_limit)) # restore

with limit(100 * (1 << 20)): # 100MiB
    # do the thing that might try to consume all memory
    i = 1
    while True:
        i <<= 1

This code consumes 100% CPU (on a single core) and the consumed memory grows very very slowly.

In principle, you should get MemoryError at some point whether it happens before your computer turns to dust is unclear. CPython uses a continuous block of memory to store the digits and therefore you may get the error even if there is RAM available but fragmented.

Your specific code shouldn't trigger it but in general you could also get OverflowError if you try to construct an integer larger than sys.maxsize bytes.

So, no spillage into pagefile then, because it would not be continuous. — mcu, Sep 20 '15 at 21:17
@coding4fun: python doesn't care where the memory comes from. Whether OS uses pagefile or not is completely transparent for python. It looks like the algorithm is too slow to fill the memory anyway. — jfs, Sep 20 '15 at 21:41

Python Infinite Integers

1 Answers1