2

How can I free part of list's memory in python? Can I do it in the following manner:

del list[0:j]

or for single list node:

del list[j]

Mark: My script analyzes huge lists and creates huge output that is why I need immediate memory deallocation.

David
  • 733
  • 2
  • 11
  • 30
  • 1
    First one will remove first `j` items from the `list`. The second one will remove the _j+1_ th item. – mshsayem Dec 24 '17 at 08:12
  • 1
    What, exactly, are you trying to accomplish? – juanpa.arrivillaga Dec 24 '17 at 09:00
  • Perhaps you would better off using generators to create a pipeline that does the filtering you want on the fly? – Thijs van Dien Dec 24 '17 at 14:27
  • @ThijsvanDien could you please supply more information about the method you mentioned?Maybe I have to open another thread for that? – David Dec 27 '17 at 13:49
  • For that you'll need to demonstrate more concretely what you're doing to generate the lists and how you process them. An important factor, for example, is if elements can be considered in isolation or that decisions are made in relation to elements that precede or follow them. – Thijs van Dien Dec 27 '17 at 18:01
  • Possible duplicate of [How can I explicitly free memory in Python?](https://stackoverflow.com/q/1316767/608639) – jww Jan 20 '19 at 07:58

2 Answers2

4

You cannot really free memory manually in Python.

Using del decreases the reference count of an object. Once that reference count reaches zero, the object will be freed when the garbage collector is run.

So the best you can do is to run gc.collect() manually after del-ing a bunch of objects.

In these cases the best advice is usually to try and change your algorithms. For example use a generator instead of a list as Thijs suggests in the comments.

The other strategy is to throw hardware at the problem (buy more RAM). But this generally has financial and technical limits. :-)

Roland Smith
  • 42,427
  • 3
  • 64
  • 94
0

You can delete an item in list via four popular methods (including collections.deque) :

list remove() method :

remove removes the first matching value, not a specific index

remember This method does not return any value but removes the given object from the list.

example :

list_1 = [987, 'abc', 'total', 'cpython', 'abc'];
list_1.remove('abc')
print(list_1)
list_1.remove('total')
print(list_1)

output:

[987, 'total', 'cpython', 'abc']
[987, 'cpython', 'abc']

Second method is list del() method

You have to specify the index_no here 

list_1 = [987, 'abc', 'total', 'cpython', 'abc'];
del list_1[1]
print(list_1)
del list_1[-1:]
print(list_1)

output:

[987, 'total', 'cpython', 'abc']
[987, 'total', 'cpython']

Third one is list pop() method :

pop() removes and returns the last item in the list.

list_1 = [987, 'abc', 'total', 'cpython', 'abc'];
list_1.pop()
print(list_1)
list_1.pop()
print(list_1)
list_1.pop()
print(list_1)

output:

[987, 'abc', 'total', 'cpython']
[987, 'abc', 'total']
[987, 'abc']

Forth method is collections.deque

There are some more external module methods like :

You can pop values from both sides of the deque:

from collections import deque

d = deque()

d.append('1')
d.append('2')
d.append('3')
print(d)

d.popleft()
print(d)
d.append('1')
print(d)

d.pop()
print(d)

output:

deque(['1', '2', '3'])
deque(['2', '3'])
deque(['2', '3', '1'])
deque(['2', '3'])
Aaditya Ura
  • 12,007
  • 7
  • 50
  • 88
  • Does all the specified methods free memory? I am asking since my script analyzes huge lists and creates huge output that is why I must to ensure deallocation during runtime. – David Dec 24 '17 at 14:22
  • @David `deque` will, because it's essentially a linked list. A regular list is contiguous in memory and will only reallocate when its number of elements halves or doubles (rough estimate). Until it does, memory usage stays the same. Worse, the reallocation operation itself needs as much memory as the new list, because the old one can't be cleaned up until it has been copied over. Note that a linked list isn't ideal either. For example, it requires more memory per element and indexing operations are relatively costly. – Thijs van Dien Dec 24 '17 at 14:42