9

I am using Python (Canopy) extensively for Earth science application. Because my application is memory consuming, I am trying find way to erase variable that I don't need any more in my programs, I tried to use del command to erase the variable memory, but I found that space used by Canopy is still the same. Any ideas about how to erase variable completely from the memory. thanks

Kernel
  • 591
  • 12
  • 23
  • 3
    Is there a test case you could show us? Also, try `__import__("gc").collect()`. – d33tah Apr 22 '16 at 13:31
  • 2
    When using a language like Python, you really don't have any direct control over when memory is deallocated. If you need that level of control, consider writing your code in C and handling memory management yourself. – larsks Apr 22 '16 at 13:35
  • 3
    Canopy uses an ipython kernel that may keep variables for future interactive use; depending on what you are doing, restarting the kernel is sometimes a good idea to restore memory and refresh the namespace. – Reblochon Masque Apr 22 '16 at 13:39
  • 3
    `del` does not delete objects, it removes references to them. Use [`sys.getrefcount()`](https://docs.python.org/2/library/sys.html#sys.getrefcount) or [`weakrefs`](https://docs.python.org/2/library/weakref.html) to see whether there's something that is keeping an object alive – Andrea Corbellini Apr 22 '16 at 13:40
  • 2
    It's complicated. Take a look at http://stackoverflow.com/questions/15455048/releasing-memory-in-python – PM 2Ring Apr 22 '16 at 13:40
  • 2
    Yes, check out *weakrefs* and what it is used for. Then check your code for spots where you might be keeping references. Caches are an obvious point - you need them to speed up, but you don't need them logically, so ref-hoarding in them is bad. – JL Peyret Apr 22 '16 at 13:43
  • 1
    p.s. did you see http://stackoverflow.com/questions/15455048/releasing-memory-in-python?lq=1 from the Linked list? one of the answers suggests launching "big temporary stuff" via subprocesses. – JL Peyret Apr 22 '16 at 18:48

2 Answers2

8

You can't manually nuke an object from your memory in Python!

The Python Garbage Collector (GC) will automatically free up memory of objects that have no existing references any more (implementation details differ per interpreter). It's periodically checking for abandoned objects in background without your interaction.

So to get an object recycled, you have to eliminate all references to it by assigning a different value (e.g. None) to all variables that pointed to the object. You can also delete a variable name using the del statement, but as you already noticed, this only deletes the name with the reference, but not the object and its data itself. Only the GC can do that.

Byte Commander
  • 6,506
  • 6
  • 44
  • 71
  • 5
    And even then, the reclaimed memory isn't returned back to the OS immediately. Like many sophisticated programs the CPython interpreter has its own internal memory management for efficiency reasons. You can read about Python's memory arenas in the link in my comment on the question, and in [the docs](https://docs.python.org/3/c-api/memory.html). – PM 2Ring Apr 22 '16 at 13:44
  • 2
    All the responses and comments above are correct. To sum up: deleting *all* Python references to an object (including implicit references in the IPython kernel), and then forcing Python garbage collector to run will make the memory eligible for re-use in the same Python process, and will often but not always make it for re-use by the OS. But even then it won't necessarily show up in OS stats. – Jonathan March Apr 22 '16 at 16:02
  • All, thanks a lot for your help. It is clear now that erasing variable from memory is not straight forward, I think that I can apply some technique like shown here by deleting all python reference and then calling the GC, or by creating subprocess as mention above. Yet, I am using my script on different platform, thus I am not sure whether such techniques will work effectively across different platform. I have to test it. Thanks again. – Kernel Apr 22 '16 at 22:32
  • Byte Commander, did you meant in your title that ** I cannot automatically …….** – Kernel Apr 22 '16 at 22:33
2

Now, it is possible to do it manually. To do that, you need to do it in two steps:

  1. use the command del to remove the variable from the workspace
  2. use the comand gc.collect to run the garbage collector and clear the ram memory

A code example:

import numpy as np
import gc

a = np.array([1,2,3])
del a
gc.collect()

More info: link

rafaoc
  • 586
  • 7
  • 21