Python list comprehension Remove duplicates

Question

I want unique elements in hubcode_list. I can do it by

hubcode_alarm_obj = HubAlarm.objects.all()
for obj in hubcode_alarm_obj:
    hubcode = obj.hubcode
    if hubcode not in hubcode_list:
        hubcode_list.append(hubcode)

I want to use list comprehension. I'm trying this but it says hubcode_list is undefined, which is the correct bug. How can I only add unique elements in hubcode_list

hubcode_alarm_obj = HubAlarm.objects.all()
hubcode_list = [obj.hubcode for obj in hubcode_alarm_obj if obj.hubcode not in hubcode_list]

I can also do it by:-

hubcode_list = list(set([obj.hubcode for obj in hubcode_alarm_obj]))

But set is again another operation. Can I do it by using if-statement in list-comprehension?

Short answer: no. Long answer: yes in old python versions with a disgusting hack. You should do this with a `set` - even though it's a second "operation", it would be faster than the proposed list comprehension. — Tim, May 27 '14 at 05:00
Is using [`distinct()`](https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.distinct) an option? — alecxe, May 27 '14 at 05:20
Ohh yeah... I can use distinct as well. Nice suggestion. Thanks. +1 — Praful Bagai, May 27 '14 at 05:22
Thanks for the suggestion. On the other note, Can I update a column of all the filtered results in one go. Currently I do like this. `reminder_obj = HubAlarm.objects.filter(a = b) for obj in reminder_obj: obj.repeat = 'False' obj.save() ` — Praful Bagai, May 27 '14 at 05:26

score 2 · Answer 1 · answered May 27 '14 at 05:04

2

Since you are using django, let the database do the filtering for you. It will be (by far) the fastest option:

objects = HubAlarm.objects.exclude(hubcode__in=hubcode_list)

See the documentation for more.

answered May 27 '14 at 05:04

Burhan Khalid

169,990
18
245
284

Thanks for it.I've another question. Can I update a column of all the filtered results in one go. Currently I do like this. `reminder_obj = HubAlarm.objects.filter(a = b) for obj in reminder_obj: obj.repeat = 'False' obj.save()` – Praful Bagai May 27 '14 at 05:21
1

Sure, see [`update()`](https://docs.djangoproject.com/en/1.6/ref/models/querysets/#update). – Burhan Khalid May 27 '14 at 05:28

score 0 · Answer 2 · answered May 27 '14 at 05:03

Your answer is no. List comprehension creates a completely new list all at one shot, so you can't check to see if something exists:

>>> lst = [1, 2, 2, 4, 5, 5, 5, 6, 7, 7, 3, 3, 4]
>>> new = [item for item in lst if item not in new]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'new' is not defined
>>>

Therefore, the preferred one-liner is to use list(set(...)).

>>> lst = [1, 2, 2, 4, 5, 5, 5, 6, 7, 7, 3, 3, 4]
>>> list(set(lst))
[1, 2, 3, 4, 5, 6, 7]
>>>

However, in a list of lists, this would not work:

>>> lst = [[1, 2], [1, 2], [3, 5], [6, 7]]
>>> list(set(lst))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'
>>>

In that case, use the standard for loop and .append().

Python list comprehension Remove duplicates

2 Answers2

Linked