0

I have a large list containing large objects from a single class:

my_list = [ LargeClass() for i in xrange(10000)]

I need to copy a slice of the list into an auxiliary list, but then to conserve memory, I would like to replace that slice in the original list by a bunch on None's:

new_list = my_list[:1000]
my_list[:1000] = [None] * 1000

My hope was that this would reduce the memory used by 'my_list' so I don't carry around two copies of the same "data". However, this doesn't release any memory. Calling the garbage collector doesn't make any difference either.

Is there any way to accomplish this?

Edit: I should have mentioned that the second list will be passed as an argument to a child process (multiprocessing), so it will be copied. When that's done, I don't need the data in the original list, which is just wasting memory now.

navidoo
  • 309
  • 2
  • 11
  • 1
    There's no need to do any replacing. Creating the new list won't copy any memory. Think of the variable as a label, the label will exist in both lists but both labels will be pointing to the same object. – Roger Fan Oct 03 '14 at 22:48
  • Thanks for the reply. I should've mentioned that I do this because I need to pass the second list to a child process (multiprocessing), so it *will* be copied. In the mean time, I don't want to keep the data in the original list because I don't need it, and I need to free some memory. – navidoo Oct 03 '14 at 22:58
  • let it go out of scope, or use `del` on this list. – m.wasowski Oct 03 '14 at 23:12

2 Answers2

2

As Roger said, you're not actually copying objects, you're duplicating references:

In [27]: mylist = [object() for i in range(10000)]

In [28]: newlist = mylist[:1000]

In [29]: mylist[0] is newlist[0]
Out[29]: True

is checks that the objects are the same thing (not merely equal, but the same)

If you want to destroy objects, you need to remove all references to them; which (not what you want to do, but) could be accomplished by simply stating:

mylist = mylist[:1000]

All told though, I've never had the demonstrated need to monkey with Python's GC. Either the objects are small enough where I don't care, or if they're massive (hundreds of MB) then they seem to be cleared in short order. My "problems" with garbage collection are usually problems in my code where I have a sneaky reference that I didn't clean up.

Nick T
  • 25,754
  • 12
  • 83
  • 121
  • And even if you destroy the objects, there is no guarantee that the memory is truly _free_. Python may not release it's claim on that chunk of memory until much later or never (but if it doesn't give the memory back to the OS, it'll probably reuse it for new objects...) – mgilson Oct 03 '14 at 22:58
  • @mgilson true, but I've never had the demonstrated need to monkey with Python's GC; either the objects are small enough where I don't care, or if they're massive then they seem to be cleared in short order. "Problems" with garbage collection are usually problems in my code where I have a sneaky reference that I didn't clean up. – Nick T Oct 03 '14 at 23:02
  • [This answer](http://stackoverflow.com/a/5495318/2073595) is relevant to what @mgilson notes above, about memory not necessarily being freed, even if all references are gone. – dano Oct 03 '14 at 23:23
  • @mgilson If you need that fine-tuned control of memory then garbage-collected languages are probably not for you. – Roger Fan Oct 04 '14 at 06:41
  • @RogerFan -- I absolutely agree. I just wanted to point out to OP that asking a question about how to get memory back from python is 1) implementation dependent, 2) maybe not possible, 3) possibly requires in depth analysis of the code. – mgilson Oct 04 '14 at 07:04
0

As in comment says , you cannot free the memory with changing the labels so as a better way you can remove the indexes that you don't want :

my_list[:1000] = []
Mazdak
  • 105,000
  • 18
  • 159
  • 188