-1

I'm Working on a big dataset that consist of float numbers, I can't load all of samples together because of memory limitation and I have to split them to parts and load one part and train my network after that delete it and do this for every part.

my problem is how can i delete the trained part of data and load another

Simple command del my_var doesn't release memory.

after that I tried %reset_selective "my var" in jupyter-notebook and gc.collect() Even though I can't access to that variable anymore but still the memory doesn't released.

python3

Ali Ghofrani
  • 311
  • 1
  • 9
  • 1
    Objects will be garbage collected if there is no reference to them anymore. So, you have to make sure the object is removed everywhere. – Klaus D. May 26 '19 at 11:33

1 Answers1

1

Python uses reference counting for memory management. When a variable goes out of scope (or is deleted explicitly as you are doing), the reference count of the object it points to is decremented by 1. Once the reference count reaches 0, the object is garbage collected (though you can't rely on the exact moment this happens). Your problem is likely that either you still have a name pointing to the objects in question or you may perhaps be deleting a copy of the data.

You'll need to provide a specific code example to get more specific help.

JohnO
  • 777
  • 6
  • 10