Python string comparison similarity

Question

I am trying to compare two lists of data which has some free text denoting the same object. example

List 1 ['abc LLC','xyz, LLC']
List 2 ['abc , LLC','xyz LLC']

It is a simple example but the problem is there can be many changes like changes in case or adding some "." in between. Is there any python package that can do the comparison and give a measure of similarity?

@OliCharlesworth I think the author wants to find a percentage of similarity between two strings. Like if the strings are 85% similar. — bezmax, Apr 04 '12 at 07:52
You don't want "probability", you want "similarity". http://stackoverflow.com/questions/682367/good-python-modules-for-fuzzy-string-comparison — Joe, Apr 04 '12 at 07:52
I believe you've got to define your problem more precisely: what kind of similarity are you detecting? What is the mathematical definition of your similarity? Otherwise people can only guess what you want. Or maybe that is actually your question: you want people to suggest you a similarity definition (like Levenshtein Distance)? — HongboZhu, Apr 04 '12 at 09:31
Possible duplicate of [Find the similarity metric between two strings](https://stackoverflow.com/questions/17388213/find-the-similarity-metric-between-two-strings) — NoDataDumpNoContribution, Jun 25 '19 at 08:01

score 7 · Accepted Answer · answered Apr 04 '12 at 07:54

You could use an implementation of the Levenshtein Distance algorithm for non-precise string matching, for instance this one from Wikibooks.

Another option would be to, for instance, fold everything to lower case, remove spaces, etc. prior to raw comparison -- this of course depends on your use case:

import string, unicodedata
allowed = string.letters + string.digits
def fold(s):
  s = unicodedata.normalize("NFKD", unicode(s).lower()).encode("ascii", "ignore")
  s = "".join(c for c in s if c in allowed)
  return s

for example in ['abc LLC','xyz, LLC', 'abc , LLC','xyz LLC']:
  print "%r -> %r" % (example, fold(example))

would print

'abc LLC' -> 'abcllc'
'xyz, LLC' -> 'xyzllc'
'abc , LLC' -> 'abcllc'
'xyz LLC' -> 'xyzllc'

score 3 · Answer 2 · answered Apr 04 '12 at 08:21

3

there's an excellent binary library that uses levenshtein distance (edit distance) between strings to estimate similarity. Give it a try:

https://github.com/miohtama/python-Levenshtein

answered Apr 04 '12 at 08:21

Not_a_Golfer

47,012
14
126
92

Python string comparison similarity

2 Answers2