Stop a program before it uses too much memory

Question

I'm working on a Python program which sometimes fills up a list with millions of items. The computer (Ubuntu) starts swapping and the debugger (Eclipse) becomes unresponsive.

Is it possible to add a line in the cycle that checks how much memory is being used, and interrupts the execution, so I can check what's going on?

I'm thinking about something like:

if usedmemory() > 1000000000:
    pass # with a breakpoint here

but I don't know what used memory() could be.

check out http://stackoverflow.com/questions/938733/total-memory-used-by-python-process — Joel Green, Jul 21 '13 at 00:06

score 3 · Accepted Answer · edited May 23 '17 at 11:51

This is highly dependant on the machine you're running Python on. Here's a SO answer for a way to do it on Linux https://stackoverflow.com/a/278271/541208, but the other answer there offers a more platform independant solution: https://stackoverflow.com/a/2468983/541208: The psutil library, which you can install via pip install psutil:

>>> psutil.virtual_memory()
vmem(total=8374149120L, available=2081050624L, percent=75.1, used=8074080256L, free=300068864L, active=3294920704, inactive=1361616896, buffers=529895424L, cached=1251086336)
>>> psutil.swap_memory()
swap(total=2097147904L, used=296128512L, free=1801019392L, percent=14.1, sin=304193536, sout=677842944)

So you'd look at the percent of the available memory and kill your process depending on how much memory it has been using

Stop a program before it uses too much memory

1 Answers1