Python function to merge unique values form multiple lists to one list

Question

I am pretty new to Python. I am trying to write a function that will merge unique values in separate lists into one list. I keep getting a result of a tuple of lists. Ultimately I would like to have one list of unique values from my three lists -a,b,c. Can anyone give me a hand with this?

def merge(*lists):
    newlist = lists[:]
    for x in lists:
        if x not in newlist:
            newlist.extend(x)
    return newlist

a = [1,2,3,4]
b = [3,4,5,6]
c = [5,6,7,8]

print(merge(a,b,c))

I am getting a Tuple of Lists

([1, 2, 3, 4], [3, 4, 5, 6], [5, 6, 7, 8])

For order-preserving output, consider [this answer](https://stackoverflow.com/a/68626841/). — Asclepius, Aug 02 '21 at 19:21

score 22 · Answer 1 · answered Apr 04 '13 at 04:42

22

You may just need sets:

>>> a = [1,2,3,4]
>>> b = [3,4,5,6]
>>> c = [5,6,7,8]
>>>
>>> uniques = set( a + b + c )
>>> uniques
set([1, 2, 3, 4, 5, 6, 7, 8])
>>>

answered Apr 04 '13 at 04:42

g.d.d.c

46,865
9
101
111

score 5 · Answer 2 · answered Apr 04 '13 at 04:54

If you don't care about them being in the original order, the easiest and likely fasted way is to use set functions:

>>> set().union(a, b, c)
{1, 2, 3, 4, 5, 6, 7, 8}

If you do care about the original order (sets happen to preserve it in this case, but aren't guaranteed to), then you can fix your original attempt by realising that the argument lists contains a tuple of all of the original lists you passed in. This means that iterating over it gets you each of those lists one at a time, rather than the elements in them - you can fix this by using the itertools module:

for x in itertools.chain.from_iterable(lists):
   if x not in newlist:
      newlist.append(x)

Also, you would want newlist to start out as an empty list rather than a copy of the input lists.

+1 for using union instead of adding the lists together beforehand. This means that you can use other iterables instead of just lists. — daboross, Apr 08 '14 at 06:09
Note that depending on the number of unique values the time to check `x not in newlist` will grow for each iteration. — g.d.d.c, Sep 19 '14 at 17:04

score 1 · Answer 3 · answered Aug 09 '18 at 23:38

Handling a Dynamically Generated List of Lists

A common use case is generating a list of lists dynamically, with each child list sometimes having an arbitrary length:

import random
abc, values = [], ["a", "b", "c", "d"]
for i in range(3):
    l = []
    for j in range(3):
        l.append(values[random.randint(0, len(values) - 1)])
    abc.append(l)

If you're working with a list of lists, simply summing them as proposed by g.d.d.c. doesn't work. That is:

uniques = set( a + b + c )

The hiccup comes from the fact that you have to specifically reference the lists a, b, and c. The answer by Ivc is really nice and gets us closer:

set().union(a, b, c)

But again, you have to explicitly reference your lists.

The Solution

To get the unique values from a list of lists with arbitrary lengths, you can use positional expansion:

import random
abc, values = [], ["a", "b", "c", "d"]
for i in range(3):
    l = []
    for j in range(3):
        l.append(values[random.randint(0, len(values) - 1)])
    abc.append(l)
# The Important Line Below
unique = set().union(*abc)
print(unique)

Which will return the appropriate values unordered (e.g. ["d", "b", "a", "d"])

user2243215 · Answer 4 · 2013-04-07T05:28:49.770

0

def merge(*lists):
    newlist = []
    for i in lists:
            newlist.extend(i)
    return newlist

merge_list = merge(a,b,c,d)

merge_list = set(merge_list)

merge_list = list(merge_list)

print(merge_list)

edited Apr 07 '13 at 05:28

answered Apr 07 '13 at 05:22

user2243215

231
2
3
5

Python function to merge unique values form multiple lists to one list

4 Answers4