How to find unique starts of strings?

Question

If I have a list of strings (eg 'blah 1', 'blah 2' 'xyz fg','xyz penguin'), what would be the best way of finding the unique starts of strings ('xyz' and 'blah' in this case)? The starts of strings can be multiple words.

Answer 1

Your question is confusing, as it is not clear what you really want. So I'll give three answers and hope that one of them at least partially answers your question.

1. To get all unique prefixes of a given list of string, you can do:

   >>> l = ['blah 1', 'blah 2', 'xyz fg', 'xyz penguin']
   >>> set(s[:i] for s in l for i in range(len(s) + 1))
   {'', 'xyz pe', 'xyz penguin', 'b', 'xyz fg', 'xyz peng', 'xyz pengui', 'bl', 'blah 2', 'blah 1', 'blah', 'xyz f', 'xy', 'xyz pengu', 'xyz p', 'x', 'blah ', 'xyz pen', 'bla', 'xyz', 'xyz '}
   

   This code generates all initial slices of every string in the list and passes these to a set to remove duplicates.

2. To get all largest initial word sequences smaller than the full string, you could go with:

   >>> l = ['a b', 'a c', 'a b c', 'b c']
   >>> set(s.rsplit(' ', 1)[0] for s in l)
   {'a', 'a b', 'b'}
   

   This code creates a set by splitting all strings at their rightmost space, if available (otherwise the while string will be returned).

3. On the other hand, to get all unique initial word sequences without considering full strings, you could go for:

   >>> l = ['a b', 'a c', 'a b c', 'b c']
   >>> set(' '.join(w[:i]) for s in l for w in (s.split(),) for i in range(len(w)))
   {'', 'a', 'b', 'a b'}
   

   This code splits each word at any whitespace and concatenates all initial slices of the resulting list, except the largest one. This code has pitfall: it will e.g. convert tabs to spaces. This may or may not be an issue in your case.

Answer 2

If you mean unique first words of strings (words being separated by space), this would be:

arr=['blah 1', 'blah 2' 'xyz fg','xyz penguin']
unique=list(set([x.split(' ')[0] for x in arr]))