Transforming unicode characters to a string containing their u+[hexa] representation ("\u2030")

Question

I am working with java 8 and I18N. From my understandings, the .properties files (and subsequent I18N code) asumes that the files are in the "ISO-8859-1" file format. Thus I'm having trouble with characters that cannot be represented in that file format.

Changing from a file writer to an OutputStreamWriter won't help since the other end of the code won't be able to read these characters anyway.

I did come up with a solution that works, but it is highly inelegant.

StringBuilder utfRepresentation = new StringBuilder();
for (int index = 0; index < input.length(); index++) {
    if (!Charset.forName("ISO-8859-1").newEncoder().canEncode(input.charAt(index))) {
        utfRepresentation.append("\\u");
        utfRepresentation.append(Integer.toHexString(input.codePointAt(index)));
    } else {
        utfRepresentation.append(input.charAt(index));
    }
}

Now I do need to do other things like extract the encoder instead of making a new one every time, but my question is something else entirely:

1) Is there a cleaner way of transforming ‰ into \u2030
2) What even is this U+2030? UTF-8/16?
3) Is there a better way of creating that charset / encoder? Something that isn't static? can I extract it from the file? or a file reader / writer?

Answer 1

As a historical anomaly, .properties are in ISO-8859-1, for which you can use StandardCharsets.ISO_8859_1 (if not on Android).

However you may use for other characters the u-escaping: \u2030 where one should understand that this is a representation of UTF-16 as stored in a single char (two bytes). Some Unicode symbols exceed the two byte limit, and are encoded in a "surrogate" pair.

• When reading from a PropertyResourceBundle, every \uXXXX will be automatically decoded
• You could the build convert a UTF-8 template file into u-escaped .properties; for instance in maven.
• Sometimes a ListResourceBundle is a better fit. It has an array in java, all java sources could be set to UTF-8 for an international project. Its behavior is different: all strings are loaded immediately.

However evidently you also want to write to .properties in code; hence not on the class path.

Here best seems Properties

For that the Properties class is ideal. It has an XML variant (instead of key-value lines) for the properties, which by default use UTF-8. But also traditional .properties can be read and written in another (UTF-8) encoding.

---

StringBuilder utfRepresentation = new StringBuilder();
for (int index = 0; index < input.length(); index++) {
    char ch = input.charAt(index);
    if (ch < 128) {
        utfRepresentation.append(ch);
    } else {
        utfRepresentation.append(String.format("\\u%04X", ch));
    }
}