Separating Unicode ligature characters

Question

Throughout the vast number of unicode characters, there are some that actually represent more than one character, like the U+FB00 ligature ﬀ for two 'f' characters. Is there any way easy to convert characters like these into multiple single characters? Preferably something available in the standard Java API, but I can refer to an external library if need be.

I took the freedom to add the keyword *ligature* to your question. :) — deceze, Aug 24 '11 at 06:41

score 25 · Accepted Answer · answered Aug 24 '11 at 07:31

25

U+FB00 is a compatibility character. Normally Unicode doesn't support separate codepoints for ligatures (arguing that it's a layout decision if and when a ligature should be used and should not influence how the data is stored). A few of those still exist to allow round-trip conversion compatibility with older encodings that do represent ligatures as separate entities.

Luckily, the information which characters the ligature represents is present in the Unicode data file and most capable string handling systems have that data built-in.

In Java, you'll need to use the Normalizer class and the NFKC form:

String ff ="\uFB00";
String normalized = Normalizer.normalize(ff, Form.NFKC);
System.out.println(ff + " = " + normalized);

This will print

ﬀ = ff

answered Aug 24 '11 at 07:31

Joachim Sauer

302,674
57
556
614

10

@nonoitall: NFKD is no panacea: there are plenty of ligatures and other notionally combined forms **it just does not work on at all.** For example, it will not manage to decompose *ß* or *ẞ* to *SS* (even those there is a casefold thither!), nor *Æ* to *AE* or *æ* to *ae*, nor *Œ* to *OE* or *œ* to *oe*. It is also useless for turning *ð* or *đ* into *d* or *ø* into *o*. For **all** those things, you need the UCA (Unicode Collation Algorithm), **not** NFKD. NFD/NFKD also both have the annoying property of destroying singletons, if this matters to you. – tchrist Aug 24 '11 at 19:59
7

@tchrist: my understanding is that those decompositions you mention *should* not be done. They are not simply ligatures in the typographical sense, but real separate characters that are used differently! ß *can* be decomposed to ss if necessary (for example if you can only store ASCII), but they are **not** equivalent. The ff Ligature, on the other hand is *only* a typographical ligature. – Joachim Sauer Aug 25 '11 at 04:33
5

@tchrist, how did you apply UCA in java, is there an example of how to break down œ to oe for example? – ledlogic Aug 31 '16 at 00:07

score 5 · Answer 2 · answered Aug 24 '11 at 07:36

The process you are talking about is called Normalization and is specified in the Unicode Normalization Forms technical note.

There is a class in the Java SE class library called java.text.Normalizer which implements this process. However, you need to read the Unicode document linked above to figure out which of the "normalization forms" you need to use to get the result you want. It is not straightforward ....

score 1 · Answer 3 · answered Aug 24 '11 at 07:25

1

You could try the java.text.Normalizer, but I am not really sure if that works for ligatures.

answered Aug 24 '11 at 07:25

fstab

992
1
11
19

Separating Unicode ligature characters

3 Answers3

Linked