How to remove repeated array elements and keep first occurrence in python?

Question

I have the following numpy array with size (5,2):

A = [[5,6]
     [4,3]
     [2,1]
     [4,3]
     [8,9]]

I want to remove any repeated rows (so in this case [4,3]) and keep the first occurrence and return what looks like:

A = [[5,6]
     [4,3]
     [2,1]
     [8,9]]

Redgen · Answer 1 · 2020-06-25T21:27:30.720

2

Function numpy.unique can help you to extract unique values from an array. Specify the axis parameter to choose the axis to operate on. Try this

np.unique(A, axis=0)

to extract unique rows.
numpy.unique will return a sorted array. If you want to retrieve original order, you can use return_index=True parameter to return indices of first appearances of the unique elements, and then get these elements from the original array. For your example, this will work:

A[np.sort(np.unique(a, return_index=True, axis=0)[1])]

edited Jun 25 '20 at 21:27

answered Jun 25 '20 at 01:30

Redgen

129
1
6

Tried it on a more complicated example and it doesn't remove them repeated rows or keep the order – lceans Jun 25 '20 at 15:55
@Iceans I'm sorry for missleading you. np.unique doesn't keep the order, it returns elements in sorted order. I updated my answer with an idea of how you can keep the order. – Redgen Jun 25 '20 at 21:22
@Iceans Also, make sure that your axis parameter is right. If it still doesn't work, please bring on your more complicated example. – Redgen Jun 25 '20 at 21:31
@redgen is there a way to message you some code? I still cant get it to work – lceans Jun 25 '20 at 22:34
@Iceans unfortunatelly, there are no private messages on stackoverflow. You can edit your question and add this code or add some external link. I would be glad to help you. – Redgen Jun 26 '20 at 09:20

Avinash Dalvi · Answer 2 · 2020-06-25T01:32:34.983

0

A = [[5,6],
     [4,3],
     [2,1],
     [4,3],
     [8,9]]
B = set(map(tuple, A))
print(B)

without loosing order :

uniq = []
for i in A:
    if not i in uniq:
        uniq.append(i)
print('l=%s' % str(A))
print('uniq=%s' % str(uniq))

edited Jun 25 '20 at 01:32

answered Jun 25 '20 at 01:27

Avinash Dalvi

8,551
7
27
53

It's too bad you loose the order of the list. – Mark Jun 25 '20 at 01:30
true. i added both option. – Avinash Dalvi Jun 25 '20 at 01:32
I am working with a numpy array not a list. Arrays do not have the append feature but thanks – lceans Jun 25 '20 at 15:37

score 0 · Answer 3 · answered Jun 25 '20 at 01:37

0

Use value converted to string as key. Unwrap dict to get list of unique elements.

list(dict([[str(x),x] for x in A]).values())

answered Jun 25 '20 at 01:37

user120242

14,918
3
38
52

How to remove repeated array elements and keep first occurrence in python?

3 Answers3