-1

How to remove duplicate items in a list. For example spider gives me below output:

[
 {"category": "Movies", "id": 4},
 {"category": "Movies", "id": 5} 
 {"category": "Movies", "id": 4}
]

I want to remove 1st and last item because they are same. please help.

mcsim
  • 1,647
  • 2
  • 16
  • 35
michael
  • 21
  • 8
  • What is so specific about scrapy in your question? – mcsim Nov 28 '17 at 12:55
  • Possible duplicate of [Python - List of unique dictionaries](https://stackoverflow.com/questions/11092511/python-list-of-unique-dictionaries) – mcsim Nov 28 '17 at 12:58

1 Answers1

0

x will be your list

uniquex = []
deleted = []


for i in x:
    if i in uniquex:
        uniquex.remove(i)
        deleted.append(i)

    elif i not in deleted:
        uniquex.append(i)
DisplayName
  • 87
  • 1
  • 9