1

I'm looking for a way to compare string values where certain characters within the strings may be punctuated characters such as é or ô, however, the puncuation should be disregarded. For example, when searching a list of names, the user might provide criteria Rene which should match list entries Rene and René (i.e. ASCII 101 and 233 should be regarded as the same thing).

Thanx

EDIT: Preferably across all UNICODE characters. I suppose one could [should?] implement some custom solution for this; I was just wondering if there is something that already exist - almost like Char.GetBaseCharacterFromPunctuatedCharacter(char) :P

that0th3rGuy
  • 1,356
  • 1
  • 15
  • 19

1 Answers1

1

You did not say which language you are using so I answer using java. Other languages have similar constructs. Also, you mean diacritics, not punctuation (.,?!...)

The collator class supports the strength of comparison. For example, for Czech, difference in diacritics is considered to be a secondary difference.

Or you might want to remove any diacritics prior comparison by unicode canonical decomposition (decomposing all characters into basic letters + diacritics) and then removing the diacritics by a simple regex (see this SO question for an explanation of InCombiningDiacriticalMarks). In java:

public static String removeDiacritics(String str) {
  return Normalizer
     .normalize(string, Form.NFD)   // decompose into letters+diacritics
     .replaceAll("\\p{InCombiningDiacriticalMarks}+", ""); // remove diacritics
}
Community
  • 1
  • 1
Jirka
  • 4,184
  • 30
  • 40
  • Hehe, so the actual issue here was that I didn't actually know what it was called, so I didn't know what to look for. Normalization works but straight comparison is also possible, both illustrated here: http://stackoverflow.com/questions/359827/ignoring-accented-letters-in-string-comparison – that0th3rGuy Aug 07 '13 at 06:05
  • Yes, the collator I mention above can do straight comparison in java corresponding to the c# code you link to. (If you agree with my answer, could you upvote it?) – Jirka Aug 07 '13 at 12:15