7

Say that I would encode a Java character array (char[]) instance as bytes:

  • using two bytes for each character
  • using big endian encoding (storing the most significant 8 bits in the leftmost and the least significant 8 bits in the rightmost byte)

Would this always create a valid UTF-16BE encoding? If not, which code points will result in an invalid encoding?

This question is very much related to this question about the Java char type and this question about the internal representation of Java strings.

Community
  • 1
  • 1
Maarten Bodewes
  • 90,524
  • 13
  • 150
  • 263

1 Answers1

9

No. You can create char instances that contain any 16-bit value you desire---there is nothing that constrains them to be valid UTF-16 code units, nor constrains an array of them to be a valid UTF-16 sequence. Even String does not require that its data be valid UTF-16:

char data[] = {'\uD800', 'b', 'c'};  // Unpaired lead surrogate
String str = new String(data);

The requirements for valid UTF-16 data are set out in Chapter 3 of the Unicode Standard (basically, everything must be a Unicode scalar value, and all surrogates must be correctly paired). You can test if a char array is a valid UTF-16 sequence, and turn it into a sequence of UTF-16BE (or LE) bytes, by using a CharsetEncoder:

CharsetEncoder encoder = Charset.forName("UTF-16BE").newEncoder();
ByteBuffer bytes = encoder.encode(CharBuffer.wrap(data)); // throws MalformedInputException

(And similarly using a CharsetDecoder if you have bytes.)

一二三
  • 21,059
  • 11
  • 65
  • 74
  • *If* the `String` contains only valid Unicode, then `toCharArray()` is necessarily valid UTF-16BE---but, as I mentioned, `String` doesn't validate its contents, so it's up to you to check this. – 一二三 Jul 25 '15 at 11:24