I'm trying to find a short way to see if any of the following items is in a list, but my first attempt does not work. Besides writing a function to accomplish this, is the any short way to check if one of multiple items is in a list.
>>> a = [2,3,4]
>>> print (1 or 2) in a
False
>>> print (2 or 1) in a
True
>>> L1 = [2,3,4]
>>> L2 = [1,2]
>>> [i for i in L1 if i in L2]
[2]
>>> S1 = set(L1)
>>> S2 = set(L2)
>>> S1.intersection(S2)
set([2])
Both empty lists and empty sets are False, so you can use the value directly as a truth value.
I was thinking of this slight variation on Tobias' solution:
>>> a = [1,2,3,4]
>>> b = [2,7]
>>> any(x in a for x in b)
True
Maybe a bit more lazy:
a = [1,2,3,4]
b = [2,7]
print any((True for x in a if x in b))
Think about what the code actually says!
>>> (1 or 2)
1
>>> (2 or 1)
2
That should probably explain it. :) Python apparently implements "lazy or", which should come as no surprise. It performs it something like this:
def or(x, y):
if x: return x
if y: return y
return False
In the first example, x == 1 and y == 2. In the second example, it's vice versa. That's why it returns different values depending on the order of them.
a = {2,3,4}
if {1,2} & a:
pass
Code golf version. Consider using a set if it makes sense to do so. I find this more readable than a list comprehension.
1 line without list comprehensions.
>>> any(map(lambda each: each in [2,3,4], [1,2]))
True
>>> any(map(lambda each: each in [2,3,4], [1,5]))
False
>>> any(map(lambda each: each in [2,3,4], [2,4]))
True
Best I could come up with:
any([True for e in (1, 2) if e in a])
In python 3 we can start make use of the unpack asterisk. Given two lists:
bool(len({*a} & {*b}))
When you think "check to see if a in b", think hashes (in this case, sets). The fastest way is to hash the list you want to check, and then check each item in there.
This is why Joe Koberg's answer is fast: checking set intersection is very fast.
When you don't have a lot of data though, making sets can be a waste of time. So, you can make a set of the list and just check each item:
tocheck = [1,2] # items to check
a = [2,3,4] # the list
a = set(a) # convert to set (O(len(a)))
print [i for i in tocheck if i in a] # check items (O(len(tocheck)))
When the number of items you want to check is small, the difference can be negligible. But check lots of numbers against a large list...
tests:
from timeit import timeit
methods = ['''tocheck = [1,2] # items to check
a = [2,3,4] # the list
a = set(a) # convert to set (O(n))
[i for i in tocheck if i in a] # check items (O(m))''',
'''L1 = [2,3,4]
L2 = [1,2]
[i for i in L1 if i in L2]''',
'''S1 = set([2,3,4])
S2 = set([1,2])
S1.intersection(S2)''',
'''a = [1,2]
b = [2,3,4]
any(x in a for x in b)''']
for method in methods:
print timeit(method, number=10000)
print
methods = ['''tocheck = range(200,300) # items to check
a = range(2, 10000) # the list
a = set(a) # convert to set (O(n))
[i for i in tocheck if i in a] # check items (O(m))''',
'''L1 = range(2, 10000)
L2 = range(200,300)
[i for i in L1 if i in L2]''',
'''S1 = set(range(2, 10000))
S2 = set(range(200,300))
S1.intersection(S2)''',
'''a = range(200,300)
b = range(2, 10000)
any(x in a for x in b)''']
for method in methods:
print timeit(method, number=1000)
speeds:
M1: 0.0170331001282 # make one set
M2: 0.0164539813995 # list comprehension
M3: 0.0286040306091 # set intersection
M4: 0.0305438041687 # any
M1: 0.49850320816 # make one set
M2: 25.2735087872 # list comprehension
M3: 0.466138124466 # set intersection
M4: 0.668627977371 # any
The method that is consistently fast is to make one set (of the list), but the intersection works on large data sets the best!
I collected several of the solutions mentioned in other answers and in comments, then ran a speed test. not set(a).isdisjoint(b) turned out the be the fastest, it also did not slowdown much when the result was False.
Each of the three runs tests a small sample of the possible configurations of a and b. The times are in microseconds.
Any with generator and max
2.093 1.997 7.879
Any with generator
0.907 0.692 2.337
Any with list
1.294 1.452 2.137
True in list
1.219 1.348 2.148
Set with &
1.364 1.749 1.412
Set intersection explcit set(b)
1.424 1.787 1.517
Set intersection implicit set(b)
0.964 1.298 0.976
Set isdisjoint explicit set(b)
1.062 1.094 1.241
Set isdisjoint implicit set(b)
0.622 0.621 0.753
import timeit
def printtimes(t):
print '{:.3f}'.format(t/10.0),
setup1 = 'a = range(10); b = range(9,15)'
setup2 = 'a = range(10); b = range(10)'
setup3 = 'a = range(10); b = range(10,20)'
print 'Any with generator and max\n\t',
printtimes(timeit.Timer('any(x in max(a,b,key=len) for x in min(b,a,key=len))',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('any(x in max(a,b,key=len) for x in min(b,a,key=len))',setup=setup2).timeit(10000000))
printtimes(timeit.Timer('any(x in max(a,b,key=len) for x in min(b,a,key=len))',setup=setup3).timeit(10000000))
print
print 'Any with generator\n\t',
printtimes(timeit.Timer('any(i in a for i in b)',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('any(i in a for i in b)',setup=setup2).timeit(10000000))
printtimes(timeit.Timer('any(i in a for i in b)',setup=setup3).timeit(10000000))
print
print 'Any with list\n\t',
printtimes(timeit.Timer('any([i in a for i in b])',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('any([i in a for i in b])',setup=setup2).timeit(10000000))
printtimes(timeit.Timer('any([i in a for i in b])',setup=setup3).timeit(10000000))
print
print 'True in list\n\t',
printtimes(timeit.Timer('True in [i in a for i in b]',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('True in [i in a for i in b]',setup=setup2).timeit(10000000))
printtimes(timeit.Timer('True in [i in a for i in b]',setup=setup3).timeit(10000000))
print
print 'Set with &\n\t',
printtimes(timeit.Timer('bool(set(a) & set(b))',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('bool(set(a) & set(b))',setup=setup2).timeit(10000000))
printtimes(timeit.Timer('bool(set(a) & set(b))',setup=setup3).timeit(10000000))
print
print 'Set intersection explcit set(b)\n\t',
printtimes(timeit.Timer('bool(set(a).intersection(set(b)))',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('bool(set(a).intersection(set(b)))',setup=setup2).timeit(10000000))
printtimes(timeit.Timer('bool(set(a).intersection(set(b)))',setup=setup3).timeit(10000000))
print
print 'Set intersection implicit set(b)\n\t',
printtimes(timeit.Timer('bool(set(a).intersection(b))',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('bool(set(a).intersection(b))',setup=setup2).timeit(10000000))
printtimes(timeit.Timer('bool(set(a).intersection(b))',setup=setup3).timeit(10000000))
print
print 'Set isdisjoint explicit set(b)\n\t',
printtimes(timeit.Timer('not set(a).isdisjoint(set(b))',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('not set(a).isdisjoint(set(b))',setup=setup2).timeit(10000000))
printtimes(timeit.Timer('not set(a).isdisjoint(set(b))',setup=setup3).timeit(10000000))
print
print 'Set isdisjoint implicit set(b)\n\t',
printtimes(timeit.Timer('not set(a).isdisjoint(b)',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('not set(a).isdisjoint(b)',setup=setup1).timeit(10000000))
printtimes(timeit.Timer('not set(a).isdisjoint(b)',setup=setup3).timeit(10000000))
print
In some cases (e.g. unique list elements), set operations can be used.
>>> a=[2,3,4]
>>> set(a) - set([2,3]) != set(a)
True
>>>
Or, using set.isdisjoint(),
>>> not set(a).isdisjoint(set([2,3]))
True
>>> not set(a).isdisjoint(set([5,6]))
False
>>>
This will do it in one line.
>>> a=[2,3,4]
>>> b=[1,2]
>>> bool(sum(map(lambda x: x in b, a)))
True
A lot of similar answers, but for me the simplest and cleanest way is just to logical AND two sets and return the result as a bool, and I think this one is not listed yet:
def check_for_matching_elements(list1, list2):
return bool(set(list1) & set(list2))
I have to say that my situation might not be what you are looking for, but it may provide an alternative to your thinking.
I have tried both the set() and any() method but still have problems with speed. So I remembered Raymond Hettinger said everything in python is a dictionary and use dict whenever you can. So that's what I tried.
I used a defaultdict with int to indicate negative results and used the item in the first list as the key for the second list (converted to defaultdict). Because you have instant lookup with dict, you know immediately whether that item exist in the defaultdict. I know you don't always get to change data structure for your second list, but if you are able to from the start, then it's much faster. You may have to convert list2 (larger list) to a defaultdict, where key is the potential value you want to check from small list, and value is either 1 (hit) or 0 (no hit, default).
from collections import defaultdict
already_indexed = defaultdict(int)
def check_exist(small_list, default_list):
for item in small_list:
if default_list[item] == 1:
return True
return False
if check_exist(small_list, already_indexed):
continue
else:
for x in small_list:
already_indexed[x] = 1
Simple.
_new_list = []
for item in a:
if item in b:
_new_list.append(item)
else:
pass
