0

For my python script I need to read in big arrays of up to a billion elements. I read one array in and process it. After processing I do not need the input anymore. Then I read the second array in and process it and again don't need the input anymore. Is it necessary to explicitly free arrays I won't ever call again after processing it or is this done automatically by the garbage collection?

At the moment I only have 4 arrays and not yet any memory issues, will this become problematic if I have 100 of those arrays?

Jay_ESE
  • 45
  • 1
  • 7

1 Answers1

1

Have a read over this article on Garbage Collection in Python. It states that you do not explicitly have to free the memory, as this is done automatically.

However with such large objects, you may need to take a bit more care. Use the del keyword to remove all references to an object, making it no longer usable and marking it as collectible to the garbage collector.

Look at the answers to this question, they go into detail about when to use the del keyword.

For example:

# Get the data.
someList = get_list()

# Do your processing.
do_something_with(someList)

# Free the reference to the list.
del someList

Alternatively, instead of deleting and reallocating memory for the list, you could reuse the variable. This is likely to increase the speed of your program due to a decrease in memory allocation.

# Get the data.
someList = get_list()

# Do your processing.
do_something_with(someList)

# Get the data.
someList = get_next_list()

# Do your processing.
do_something_with(someList)

# Free the reference to the list.
del someList
Charlie
  • 2,876
  • 19
  • 26