1

I am trying to delete keys from a dictionary according to Python reclaiming memory after deleting items in a dictionary . One solution proposed in the latter is:

to create a new dictionary by copying the old dictionary

In source code 1 I did the following:

  • I create a new dictionary new_flights
  • delete the key del flights[key]
  • finally use new_flights = copy.deepcopy(flights)

In source code 2 I did the following:

  • I create a new dictionary new_flights
  • copy to new dictionary new_flights[key] = flights[key]
  • finally flights.clear()

Both ways didn't work. The memory is not being freed.

Source code 1

 def remove_old_departure_dates(flights, search_date):
        new_flights = defaultdict(lambda : defaultdict(list))
        s_date = datetime.strptime(search_date,'%m/%d/%y')
        
        for key in flights.keys():
            dep_date = datetime.strptime(key.DDATE,'%m/%d/%y')
            if(s_date > dep_date):
                del flights[key]

        new_flights = copy.deepcopy(flights)
        flights.clear()
        return new_flights

Source code 2

def remove_old_departure_dates(flights, search_date):
    new_flights = defaultdict(lambda : defaultdict(list))
    s_date = datetime.strptime(search_date,'%m/%d/%y')
    
    for key in flights.keys():
        dep_date = datetime.strptime(key.DDATE,'%m/%d/%y')
        if(s_date > dep_date):
            new_flights[key] = flights[key]

    flights.clear()
    return new_flights

Source code after discussion

Based on the comments I did the following and used a deepcopy to remove the reference. Seems that it's working. Is that something correct?

def remove_old_departure_dates(flights, search_date):
    old_flights = defaultdict(lambda : defaultdict(list))
    new_flights = defaultdict(lambda : defaultdict(list))
    s_date = datetime.strptime(search_date,'%m/%d/%y')
    
    for key in flights.keys():
        dep_date = datetime.strptime(key.DDATE,'%m/%d/%y')
        if(s_date > dep_date):
            print "removing dates " + key.DDATE +" "+search_date
            old_flights[key] = copy.deepcopy(flights[key])
        else:
            new_flights[key] = copy.deepcopy(flights[key])

    return (old_flights, new_flights)
Community
  • 1
  • 1
Hani Gotc
  • 840
  • 11
  • 24
  • I know I mentioned it as reference in my question. I actually applied the solution. But still didn't work. – Hani Gotc Jun 20 '17 at 18:01
  • 1
    Did you *read* the answers to the question you linked to? Doing `flights.clear()` empties the dictionary, but the dictionaries don't resize downwards. You now have a very large hash table with dummy entries. Instead, make a copy, and reassign it to the name, making sure there are no references to the old dictionary, so that the old dictionary gets garbage collected. Also, as that question states, the Python process might not be returning the freed memory to the OS. – juanpa.arrivillaga Jun 20 '17 at 18:01
  • @juanpa.arrivillaga how how can I do **making sure there are no references to the old dictionary, so that the old dictionary gets garbage collected. –** I really don't understand that part – Hani Gotc Jun 20 '17 at 18:02
  • what? No. CPython uses reference counting for memory management. When an object's reference count reaches 0, the object's memory is deallocated. – juanpa.arrivillaga Jun 20 '17 at 18:06
  • Also, what exactly is not working? You need to properly specify your problem. – juanpa.arrivillaga Jun 20 '17 at 18:06
  • @juanpa.arrivillaga yes sorry sheesh. I wanted to ask If i am properly applying what was answerd in Python reclaiming memory after deleting items in a dictionary. And why it's not working. – Hani Gotc Jun 20 '17 at 18:09
  • 1
    Again, you need to properly specify your problem. **What** isn't working? How do you know? What exactly is the problem? You've merely stated "both ways didn't work", but that isn't an adequate problem specification. – juanpa.arrivillaga Jun 20 '17 at 18:10
  • the memory is not being freed – Hani Gotc Jun 20 '17 at 18:12
  • 1
    How do you know? What are you using to monitor your memory? What sort of system are you on? Windows? OSX? Linux? Again. You need to be providing these details *in the question itself*. As stated in that question, some OSs don't "see" freed memory from a Python process, even if the memory is actually freed. – juanpa.arrivillaga Jun 20 '17 at 18:14
  • It's hard to understand what you are asking. Again, you **still have not specified your problem**. But `deepcopy` does **not** remove references. Fundamentally, you have function definitions, but we cannot know if this is adequate to remove any references to your object. Python doesn't have pass-by-reference semantics. Nothing you do inside a function affects references outside that function, unless you are working with a variable that has been marked as `global`. But even then, you might have created references to that object somewhere else in your code. It's impossible to tell. – juanpa.arrivillaga Jun 20 '17 at 18:23
  • I think I GET IT! https://stackoverflow.com/questions/5844672/delete-an-item-from-a-dictionary AHA!!!!!!!. Thank you @juanpa.arrivillaga For the time. You helped me understand how it's working. Sorry again – Hani Gotc Jun 20 '17 at 18:30
  • 1
    `x = [1,2,3]`, then `del x`. But that doesn't mean that if somewhere else in your code you did `y = x` that `y` is removed. The underlying list will continue to exist as long as there is a reference to it somewhere. – juanpa.arrivillaga Jun 20 '17 at 18:30

0 Answers0