How to remove an item from a list if its in one or two (or N) other lists

Question

I have a parent list called parent_list, and two subsets via which I'm to filter the parent_list. These subsets are also python lists, and they're called filter1 and filter2.

Can I do:

final_list = [object for object in parent_list if object.pk not in filter1 or filter2]

Or will I need to do this filteration separately, as in:

intermediate_list = [object for object in parent_list if object.pk not in filter1]
final_list = [object for object in intermediate_list if object.pk not in filter2]

I couldn't explicitly find the answer from the documentation on python list comprehensions.

score 6 · Accepted Answer · edited May 23 '17 at 10:29

6

Use sets for fast look up of the items:

final_list = [object for object in parent_list if object.pk not in set(filter1 + filter2)]
#                                                                   ^

The size of the entire search space is also reduced as duplicate items are removed from the set.

Found this somewhere on SO:

Sets are implemented using hash tables. Whenever you add an object to a set, the position within the memory of the set object is determined using the hash of the object to be added. When testing for membership, all that needs to be done is basically to look if the object is at the position determined by its hash, so the speed of this operation does not depend on the size of the set. For lists, in contrast, the whole list needs to be searched, which will become slower as the list grows.

edited May 23 '17 at 10:29

Community

1
1

answered Jun 16 '16 at 19:00

Moses Koledoye

77,341
8
133
139

Two follow up questions: **1)** do I need to convert the set to a list (i.e. list(set(filter1+filter2)) for this to work **2)** this sort of a thing won't work with ValuesQuerySet would it? – Hassan Baig Jun 16 '16 at 19:05
2

Force the QuerySets to evaluate their items by slicing: `set(filter1[:] + filter2[:])`. You don't need to convert the `set` to a `list`. – Moses Koledoye Jun 16 '16 at 19:08

Mahendra Gaur · Answer 2 · 2016-06-16T18:59:10.357

1

I think you can do it as below:

final_list = [object for object in parent_list if (object.pk not in filter1) and (object.pk not in filter2)]

I recomend make two sets filter1 and filter2 with the object.pk to be filtered just bc getting an element from a set it's faster

edited Jun 16 '16 at 18:59

answered Jun 16 '16 at 18:50

Mahendra Gaur

380
2
11

I think you want to use `and`, not `or`. – John Gordon Jun 16 '16 at 18:58
Thanks Mahendra, going with Moses' answer on this one since it had some extra optimization in it. – Hassan Baig Jun 16 '16 at 19:09

score 1 · Answer 3 · answered Jun 16 '16 at 18:53

1

You can also use:

final_list = [object for object in parent_list if object.pk not in filter1+filter2]

answered Jun 16 '16 at 18:53

Nikign

379
3
9

1

Thanks Nikign, going with Moses on this one. – Hassan Baig Jun 16 '16 at 19:10

How to remove an item from a list if its in one or two (or N) other lists

3 Answers3