Faster way to remove a dictionary of phrase from a list of string using Python

Question

I have to remove a dictionary of phrase from a list of string using Python

A list of strings L1. Example: L1 = ['Programmer New York', 'Programmer San Francisco']

A dictionary of phrase L2 (all of them are more than one word). Example: L2={'New York', 'San Francisco'}

The expected output is, for each string in L1, remove substring that exists in L2. So the output will be res=['Programmer', 'Programmer'].

def foo(L1, L2):
    res = []
    print len(L1)
    for i in L1:
        for j in L2:
            if j in i:
                i = i.replace(j, "")
        res.append(i)
    return res

My current program is a brute force double for loop. But is it possible to improve the performance? Especially when L1 size is very large.

Answer 1

Try using map() and re,

import re
res = map(lambda i, j: re.sub(" "+i, '', j), L2, L1)

The double quotes before the i are there to eliminate the trailing space after programmer.

return list(res)

P.S. returning a list explicitly is only necessary if you are using Python 3. Let me know if this improves your speed at all.

Answer 2

You can use list comprehension to do so as:

l1 = ['Programmer New York', 'Programmer San Francisco']
l2=['New York', 'San Francisco']
a=[x.split(y) for x in l1 for y in l2 if y in x]
res=["".join(x) for x in a]