Getting Distinct value in Python

Question

I have a list

[<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>,
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>,
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
<Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>, 
<Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>]

I want the distinct value of this Expected Result is

[
 <Storage {'CaseID': 45L, 'PatientProfile_ID': 3L}>, 
 <Storage {'CaseID': 46L, 'PatientProfile_ID': 2L}>
]

This data is being pulled from Database. Please don't advice me to use the DISTINCT keyword on the database. Its been sorted by another column. Using distinct will remove the sorted result.

Can it be done in python. Or do i have to write a for loop to do the same thing?

MyCODE

entries=db.query("SELECT cases.ID as CaseID,PatientProfile_ID FROM \
                             `callog` ,consultation,cases WHERE callog.Consultation_ID=consultation.ID and consultation.Case_ID = cases.ID and \
                             Doctor_ID="+str(Id)+" ORDER BY callog.CallDate DESC")
            rows=entries.list()
            return rows

@thefourtheye: See my code. Thats how webpy returns a resultset. — Abhilash Cherukat, Feb 27 '15 at 06:43

Richard · Accepted Answer · 2015-02-27T07:10:49.773

1

This should handle things nicely.

def remove_dupes_keep_order(seq):
  seen = set()
  seen_add = seen.add
  return [ x for x in seq if not (x in seen or seen_add(x))]

Using seen_add speeds up the operation.

See also, this SO question.

Since the issue seems to be that your items are of type Storage, try this:

def remove_dupes_keep_order(seq):
  seen = set()
  seen_add = seen.add
  return [ x for x in seq if not (json.dumps(x, sort_keys=True) in seen or seen_add(json.dumps(x, sort_keys=True)))]

edited Feb 27 '15 at 07:10

answered Feb 27 '15 at 06:44

Richard

56,349
34
180
251

Nope, at /doctor unhashable type: 'Storage' – Abhilash Cherukat Feb 27 '15 at 06:47
Ah, well that's crappy. Are you using the features of the `Storage` object? Or can you convert it to a standard Python dictionary? – Richard Feb 27 '15 at 06:49
@AbhilashCherukat, I've updated my answer to provide an alternative possibility. – Richard Feb 27 '15 at 06:51
No much on the net, Its only said that Storage is a Fancy type of Dictionary :( – Abhilash Cherukat Feb 27 '15 at 06:53
I didn't see much either, but perhaps you can convert it back with `dict(i_am_a_storage_object)`. – Richard Feb 27 '15 at 06:54
For the Second Version of "remove_dupes_keep_order" i got at /doctor unhashable type: 'dict' – Abhilash Cherukat Feb 27 '15 at 06:59
@AbhilashCherukat, it turns out you cannot put a dict in a set, so I convert it to a JSON representation which can be used in a set. This should work now, I believe. – Richard Feb 27 '15 at 07:11
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/71947/discussion-between-abhilash-cherukat-and-richard). – Abhilash Cherukat Feb 28 '15 at 14:36

score 0 · Answer 2 · answered Feb 27 '15 at 06:48

0

Try this:

newList = list(set(oldList))

above, turns your list into a set (which removes duplicates) and then back into a list.

For more information: Refer this. See, if that helps.

answered Feb 27 '15 at 06:48

Gaurav Dave

6,838
9
25
39

Nope, at /doctor unhashable type: 'Storage' Storage part is the problem with both the answers. – Abhilash Cherukat Feb 27 '15 at 06:50

score 0 · Answer 3 · answered Feb 27 '15 at 07:07

0

After all the trial and error, I am back to For loops

def remove_dupes_keep_order(self,seq):
        Lister=[]
        for row in seq:
            Dicter={"CaseID":row["CaseID"],"PatientProfile_ID":row["PatientProfile_ID"]}
            Lister.append(Dicter)
        seen = []
        seen_add = seen.append
        return [ x for x in Lister if not (x in seen or seen_add(x))]

Snippet Courtesy: @Richard

answered Feb 27 '15 at 07:07

Abhilash Cherukat

154
12

This has a for loop you can use the accepted answer coz of a better solution it is. – Abhilash Cherukat Feb 27 '15 at 07:17

Getting Distinct value in Python

3 Answers3