How to get unique set of words from an array by removing duplicates using python

Question

Below is the array of words I'm having

l = ['Australian Cricket Team',
     'Cricket Team Australian',
     'Won Against England',
     'Against England Team']

And I want to make it That has only unique words. Below is how I want it to be after processing

['Australian', 'Cricket' ,'Team', 'Won',  'Against', 'England']

Simply I want to have unique set of words.

You've used the pandas and nltk tags. Are you actually using these libraries? If so please add your code. I'm removing the tags for now. — ddejohn, Apr 03 '22 at 03:49

score 3 · Accepted Answer · answered Apr 03 '22 at 03:41

3

You can use a loop:

l = ['Australian Criket Team', 'Cricket Team Australian', 'Won Against England', 'Against England Team']

set(w for s in l for w in s.split())

Output: {'Against', 'Australian', 'Cricket', 'Criket', 'England', 'Team', 'Won'}

Or, if order matters:

list(dict.fromkeys(w for s in l for w in s.split()))

Output: ['Australian', 'Criket', 'Team', 'Cricket', 'Won', 'Against', 'England']

functional variant

from itertools import chain
set(chain.from_iterable(map(str.split, l)))

answered Apr 03 '22 at 03:41

mozway

194,879
13
39
75

Is the output generating with this code? I dont get any output! Can you help – Nick Code Apr 03 '22 at 04:08
@Nick you need to assign to a variable, or print. Example: `out = set(w for s in l for w in s.split()) ; print(out)` – mozway Apr 03 '22 at 04:10
Yes that worked! What if I want to get the out put as below? ['Australian Criket', 'Team Cricket', 'Won Against', 'England'] – Nick Code Apr 03 '22 at 10:16
@Nick then you need to [split the output list by chunks](https://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks). – mozway Apr 03 '22 at 10:23

score 0 · Answer 2 · edited Apr 03 '22 at 03:52

0

Try this

def unique(list1):
 
    # initialize a null list
    unique_list = []
     
    # traverse for all elements
    for x in list1:
        # check if exists in unique_list or not
        if x not in unique_list:
            unique_list.append(x)
    # print list
    for x in unique_list:
        print x,

original post at GeekforGeeks https://www.geeksforgeeks.org/python-get-unique-values-list/

edited Apr 03 '22 at 03:52

Henry Ecker

34,399
18
41
57

answered Apr 03 '22 at 03:42

Scrimpy

19
2

1

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Apr 03 '22 at 05:35

How to get unique set of words from an array by removing duplicates using python

2 Answers2

functional variant