Why is this if condition working partially?

Question

I have a list of dicts, which I would like to delete all dicts where the value of few specific keys is smaller than their counterpart in another dict :

Here's my code :

for d in clean_hosting_dicts:
    if (int(str(d["cores"]) ) < int(self.query["cores"]) ) or ( float(str(d["bandwidth"]).replace(",",".") ) < int(str(self.query["bandwidth"])) ) or ( float(str(d["storage"]).replace(",",".") ) < int(str(self.query["storage"])) ) or (float(str(d["ram"]).replace(",",".") ) < int(str(self.query["ram"])) ):
        del clean_hosting_dicts[clean_hosting_dicts.index(d)]

Where clean_hosting_dicts is my list of dicts and self.query is another dict which has the same key but other values to test against them .

Here's the json for self.query :

{"hosting_type" : "vps","cores": "3", "storage" :"2", "bandwidth" : "1", "ram" : "2", "ip" : "1"}

And here's the json for the list of dicts before this code runs (unfiltered list ) :

[
  {
    "support_score": "4,5",
    "name": "Hostgator Snappy 8000",
    "ip": "2",
    "support": "chat,email,docs,phone,ticket",
    "ram": "8",
    "storage": "240",
    "frequency_rate": 3,
    "cpu": "AMD Opteron 6378",
    "reviews": "3",
    "bandwidth": "3",
    "domains": "1",
    "cores": "4",
    "os": "CentOS 66",
    "price": "39,95",
    "guarantee": "45"
  },
  {
    "support_score": "3,5",
    "name": "Bluehost Standard",
    "ip": "1",
    "support": "email,chat,doc,phone,ticket",
    "ram": "2",
    "storage": "30",
    "frequency_rate": 3,
    "cpu": "AMD Opteron 6378",
    "reviews": "3,2",
    "bandwidth": "1",
    "domains": "1",
    "cores": "2",
    "os": "CentOS 66",
    "price": "19,99",
    "guarantee": "30"
  },
  {
    "support_score": "3,5",
    "name": "Bluehost Enhanced",
    "ip": "2",
    "support": "email,chat,doc,phone,ticket",
    "ram": "4",
    "storage": "60",
    "frequency_rate": 3,
    "cpu": "AMD Opteron 6378",
    "reviews": "3,2",
    "bandwidth": "2",
    "domains": "1",
    "cores": "2",
    "os": "CentOS 66",
    "price": "29,99",
    "guarantee": "30"
  },
  {
    "support_score": "3,5",
    "name": "Bluehost Ultimate",
    "ip": "2",
    "support": "email,chat,doc,phone,ticket",
    "ram": "8",
    "storage": "240",
    "frequency_rate": 3,
    "cpu": "AMD Opteron 6378",
    "reviews": "3,2",
    "bandwidth": "4",
    "domains": "1",
    "cores": "4",
    "os": "CentOS 66",
    "price": "59,99",
    "guarantee": "30"
  },
  {
    "support_score": "4",
    "name": "iPage Business",
    "ip": "2",
    "support": "email,chat,doc,phone,ticket",
    "ram": "4",
    "storage": "90",
    "frequency_rate": 3,
    "cpu": "Unknown",
    "reviews": "3,7",
    "bandwidth": "3",
    "domains": "1",
    "cores": "2",
    "os": "CentOS 64",
    "price": "47,99",
    "guarantee": "30"
  },
  {
    "support_score": "4,5",
    "name": "InMotion vps-1000ha-s",
    "ip": "3",
    "support": "email, phone, skype, ticket,chat",
    "ram": "4",
    "storage": "75",
    "frequency_rate": 3,
    "cpu": "Unknown",
    "reviews": "4,5",
    "bandwidth": "4",
    "domains": "1",
    "cores": 5,
    "os": "CentOS 66",
    "price": "29,99",
    "guarantee": "30"
  },
  {
    "support_score": "4,5",
    "name": "InMotion vps-3000ha-s",
    "ip": "5",
    "support": "email, phone, skype, ticket,chat",
    "ram": "8",
    "storage": "260",
    "frequency_rate": 3,
    "cpu": "Unknown",
    "reviews": "4,5",
    "bandwidth": "6",
    "domains": "1",
    "cores": 5,
    "os": "CentOS 66",
    "price": "74,99",
    "guarantee": "30"
  },
  {
    "support_score": "4",
    "name": "Fatcow Business",
    "ip": "2",
    "support": "email,chat,doc,phone,ticket",
    "ram": "4",
    "storage": "90",
    "frequency_rate": 3,
    "cpu": "Unknown",
    "reviews": "3,8",
    "bandwidth": "3",
    "domains": "1",
    "cores": "2",
    "os": "CentOS 64",
    "price": "47,99",
    "guarantee": "30"
  },
  {
    "support_score": "4",
    "name": "1and1 Cloud L",
    "ip": "1",
    "support": "email,doc,phone,ticket",
    "ram": "2",
    "storage": "80",
    "frequency_rate": 3,
    "cpu": "Intel Xeon",
    "reviews": "3",
    "bandwidth": "0",
    "domains": "1",
    "cores": "2",
    "os": "CentOS 7",
    "price": "14,99",
    "guarantee": "30"
  },
  {
    "support_score": "4,5",
    "name": "Hostgator Snappy 2000",
    "ip": "2",
    "support": "chat,email,docs,phone,ticket",
    "ram": "2",
    "storage": "120",
    "frequency_rate": 3,
    "cpu": "AMD Opteron 6376",
    "reviews": "3",
    "bandwidth": "1,5",
    "domains": "1",
    "cores": "2",
    "os": "CentOS 64",
    "price": "19,95",
    "guarantee": "45"
  }
]

After this code runs it delete all dicts that don't meet requirements, except one "the one whose name is 1&1 Cloud L" , this one has "2" as a value for cores and the query["cores"] is 4 which is bigger than two (I'm converting all this values into numbers )

I don't know why it doesn't get deleted, I couldn't find anything that would help online ir by debugging via pdb .

Possible duplicate of [How to delete items from a dictionary while iterating over it?](http://stackoverflow.com/questions/5384914/how-to-delete-items-from-a-dictionary-while-iterating-over-it) — Loïc G., Jan 09 '17 at 20:37
No it is not, first I'm looking to remove a whole dict from a list not items from dict and sec it is working but the If statement is not working partially . — Anis Souames, Jan 09 '17 at 20:45
@AnisSouames, you're iterating over a `list` that you are changing at the same time. That's why your code is not working. You need to make a copy of the `list` to loop through and edit the original. — Abdou, Jan 09 '17 at 20:49
Working partially means not working. You should not modify a collection while you are iterate over — Loïc G., Jan 09 '17 at 20:56
Possible duplicate of [Remove items from a list while iterating](http://stackoverflow.com/questions/1207406/remove-items-from-a-list-while-iterating) — wwii, Jan 09 '17 at 21:13
When you delete items in a list while iterating over it the indexing gets messed up - an item at index 4 before the delete will be at index 3 after deletion of the item at index 3 - the next index in the iteration is 4 so you end up skipping an item. — wwii, Jan 09 '17 at 21:18

score 3 · Accepted Answer · answered Jan 09 '17 at 20:50

The problem is that you are deleting items from the list that you are iterating, so I recommend iterating on a different list, for instance with the expression:

for d in clean_hosting_dicts[:]:
    ...

So you iterate over an equal list, although not the same object, and delete only in the original.

Hope this helped!

Stephen Rauch · Answer 2 · 2017-01-09T21:37:04.140

1

A list comprehension can be used to pare down a list based on a conditional:

clean_hosting_dicts = [
    d for d in clean_hosting_dicts
    if not ((int(str(d["cores"])) < int(self.query["cores"])) or (
        float(str(d["bandwidth"]).replace(",", ".")) < int(
            str(self.query["bandwidth"]))) or (
        float(str(d["storage"]).replace(",", ".")) < int(
            str(self.query["storage"]))) or (
        float(str(d["ram"]).replace(",", ".")) < int(str(self.query["ram"]))))
]

In the OP's code, the problem comes when deleting items from a list as it is being iterated on. The above approach can work because a new list is built and then assigned back to the original name.

Note (from the comments) that this will build an entirely new object, so if someone has a reference to the original list, that list will not be updated.

edited Jan 09 '17 at 21:37

answered Jan 09 '17 at 20:39

Stephen Rauch

47,830
31
106
135

1

Nice solution, unless requirement is to make changes *to* `clean_hosting_dicts` and not replace it with a new object. – Scott Hunter Jan 09 '17 at 20:41
Thanks for the solution, could you explain why it didn't work in the first code ? What was wrong ? – Anis Souames Jan 09 '17 at 20:46
1

@Anis You modify a dict while you are iterating over it – Loïc G. Jan 09 '17 at 20:53

Why is this if condition working partially?

2 Answers2