What is the fastest way to filter a set of substrings from a string in python?

Question

I'm looking at a statement that looks like this:

def fn(somelongstring):
    shorterstring = somelongstring.replace('very, ','').replace('long ', '')

fn('some very, very, very, long string')

what's the most efficient method for performing this kind of operation in Python?

Some notes:

The list of replace calls is quite long, but fixed and known in advance
The long string is an argument to the function, and can get massive; it includes repetitions of the substrings
My intuition is that deletion has the opportunity to use different, faster, algorithms from replace
The chained replace calls are probably each iterating over the string. There has to be a way to do this without all those repeated iterations.

possible duplicate of [Mass string replace in python?](http://stackoverflow.com/questions/1919096/mass-string-replace-in-python) — Cédric Julien, Sep 13 '13 at 14:39

Jon Clements · Answer 1 · 2013-09-13T14:46:50.593

3

Use an re:

import re
shorterstring = re.sub('very, |long ', '', 'some very, very, very, long string')

You'll need to make sure that the substrings to replace with nothing are in descending order of length so that longer matches are replaced first.

Or, you could avoid the chained calls, and use:

reduce(lambda a, b: a.replace(b, ''), ['very, ', 'long '], s)

edited Sep 13 '13 at 14:46

answered Sep 13 '13 at 14:40

Jon Clements

Compared to OP's method this is slower. (454 us vs 2.74 ms) – Ashwini Chaudhary Sep 13 '13 at 14:43
+1 Indeed, regex will save a lot of memory when the string is huge. – Ashwini Chaudhary Sep 13 '13 at 15:22

1 Answers1