deleting duplicates in an list of objects by attribute in python

Question

I have a function for which I would like to go through a list of custom objects (with given midpoints) one at a time and take objects with unique midpoints and put them in a new list. The following code gives me an error that the list has no attribute called 'midPoint', but I am not sure how to get the code to search the list for objects with matching midpoints. Does anyone have recommendations?

class Obj():
    def __init__(self, midPoint=[0,0]):
        self.midPoint = midPoint

obj1 = Obj(midPoint = [1,1])
obj2 = Obj(midPoint = [2,2])
obj3 = Obj(midPoint = [3,3])
obj4 = Obj(midPoint = [1,1])
obj5 = Obj(midPoint = [2,2])

l = [obj1, obj2, obj3, obj4, obj5]

list_no_duplicates = []

def Delete_duplicates(list1):        
    for i in list1:
        if i.midPoint not in list_no_duplicates.midPoint:
        list_no_duplicates.append(x)

Delete_duplicates(l)
print list_no_duplicates

Possible duplicate of https://stackoverflow.com/questions/4169252/remove-duplicates-in-list-of-object-with-python — whackamadoodle3000, May 25 '17 at 22:47
`list_no_duplicates` is a list and doesn't have a `midPoint` attribute. If you wish to look at this attribute you need to have an `Object` object at hand. Also, what is `x` in the `list_no_duplicates.append(x)` line? — Garrett Kadillak, May 25 '17 at 22:50
Do you want to maintain the order of the object in the new list? — Moinuddin Quadri, May 25 '17 at 22:51
do you want to remove those with >=2 entries, or just keep 1 of them? — jeremycg, May 25 '17 at 23:04
Sorry, I meant i not x in 'list_no_duplicates.append(x)'. Order does not need to be maintained and I would just like to keep 1 copy of those that are duplicated. Thanks! — sashaks, May 26 '17 at 22:54

Moinuddin Quadri · Accepted Answer · 2017-05-25T23:04:29.073

You may use itertools.groupby along with sorted function to remove the duplicate entries from your list. For example:

>>> from itertools import groupby

>>> [next(obj) for i, obj in groupby(sorted(l, key=lambda x: x.midPoint), lambda x: x.midPoint)]
[Obj: midPoint [1, 1], Obj: midPoint [2, 2], Obj: midPoint [3, 3]]

Here, I have added the __repr__() function to your Obj class to display the value of midPoint with your class' object:

def __repr__(self):
    return 'Obj: midPoint {}'.format(str(self.midPoint))

Note: This solution won't maintain the order of elements in orginal list. The new list will be sorted based on the value of midPoint.

Below is the complete code for your reference:

from itertools import groupby

class Obj():
    def __init__(self, midPoint=[0,0]):
        self.midPoint = midPoint
    def __repr__(self):
        return 'Obj: midPoint {}'.format(str(self.midPoint))

obj1 = Obj(midPoint = [1,1])
obj2 = Obj(midPoint = [2,2])
obj3 = Obj(midPoint = [3,3])
obj4 = Obj(midPoint = [1,1])
obj5 = Obj(midPoint = [2,2])

l = [obj1, obj2, obj3, obj4, obj5]
# `print(l)` will display:
# [Obj: midPoint [1, 1], Obj: midPoint [2, 2], Obj: midPoint [3, 3], Obj: midPoint [1, 1], Obj: midPoint [2, 2]]

# New list with unique `midPoint`s
new_list = [next(obj) for i, obj in groupby(sorted(l, key=lambda x: x.midPoint), lambda x: x.midPoint)]
# `print(new_list)` will display:
# [Obj: midPoint [1, 1], Obj: midPoint [2, 2], Obj: midPoint [3, 3]]

score 1 · Answer 2 · answered May 25 '17 at 23:06

1

Use itertools.groupby

from itertools import groupby
from operator import attrgetter as ga
...
l2 = [next(g) for k, g in groupby(sorted(l, key=ga('midPoint')), key=ga('midPoint'))]

<script src="//repl.it/embed/ISOl/1.js"></script>

answered May 25 '17 at 23:06

EyuelDK

3,029
2
19
27

score 1 · Answer 3 · answered May 25 '17 at 23:45

If i understood correctly, this may work as you asked.

class Obj(): 
    def __init__(self, midPoint=[0,0]): 
        self.midPoint = midPoint

obj1 = Obj(midPoint = [1,1])
obj2 = Obj(midPoint = [2,2])
obj3 = Obj(midPoint = [3,3])
obj4 = Obj(midPoint = [1,1])
obj5 = Obj(midPoint = [2,2]) 

l = [obj1, obj2, obj3, obj4, obj5]
list_no_duplicates = [] 

# determines if midpoint already exists in any object inside list_no_duplicates
def midpoint_exists(midpoint):
    for obj in list_no_duplicates:
        if obj.midPoint == midpoint:
            return True
    return False

def Delete_duplicates(list1): 
    for obj in list1:
        # if there are no object with this midPoint, appends
        if not midpoint_exists(obj.midPoint):
            list_no_duplicates.append(obj)

Delete_duplicates(l)
print list_no_duplicates

score 0 · Answer 4 · answered May 25 '17 at 23:10

We can use a dict, with setdefault, using the key as a tuple of the midPoint and the value as a list, which we then append to.

Then we have a dict with a list for each value of midPoint, and we can iterate out the [0]th element:

def Delete_duplicates(list1):
    deduped = {}
    for i in list1:
        deduped.setdefault(tuple(i.midPoint),[]).append(i)
    return [j[0] for i,j in deduped.items()]

If you want all those with duplicates removed, we can do the same, but return only if our lists length is 1:

def Delete_duplicates(list1):
    deduped = {}
    for i in list1:
        deduped.setdefault(tuple(i.midPoint),[]).append(1)
    return [i for i in list1 if len(deduped[tuple(i.midPoint)]) == 1]

Moinuddin Quadri · Answer 5 · 2017-05-26T04:46:06.403

If you don't care about the order of the objects in the new list, you may use a dict to achieve this:

new_list = {tuple(obj.midPoint): obj  for obj in l}.values()

# content of `new_list` will be:
# [Obj: midPoint [3, 3], Obj: midPoint [1, 1], Obj: midPoint [2, 2]]

Create a dictionary with key as tuple of your midPoint value, and values as the object of Obj class. Then call dict.values() method to get the list of values i.e. list of Obj objects in your case.

However, in Python 3.x dict.values() return an object of dict_values type. In order to convert it to list, explicitly type-cast it to list as:

new_list = list(new_list)

deleting duplicates in an list of objects by attribute in python

5 Answers5