DataOutputStream: purpose of the "encoded string too long" restriction

Question

There is a strange restriction in java.io.DataOutputStream.writeUTF(String str) method, which limits the size of an UTF-8 encoded string to 65535 bytes:

    if (utflen > 65535)
        throw new UTFDataFormatException(
            "encoded string too long: " + utflen + " bytes");

It is strange, because:

there is no any information about this restriction in JavaDoc of this method
this restriction can be easily solved by copying and modifying an internal static int writeUTF(String str, DataOutput out) method of this class
there is no such restriction in the opposite method java.io.DataInputStream.readUTF().

According to the said above I can not understand the purpose of a such restriction in the writeUTF method. What have I missed or misunderstood?

It is an assumption that you would never need to send such a long string. This class dates back to Java 1.2 and many assumptions were made in the early days which are not as valid today. I have a similar method in a library of mine, but I use stop bit encoding for the length which means it uses a byte for each 7 bits of the length, so it is shorter in the common case, but can be 2 billion. — Peter Lawrey, Mar 30 '14 at 09:32

Erwin Bolwidt · Accepted Answer · 2014-03-30T08:00:28.063

The Javadoc of DataOutputStream.writeUTF states:

First, two bytes are written to the output stream as if by the writeShort method giving the number of bytes to follow. This value is the number of bytes actually written out, not the length of the string.

Two bytes means 16 bits: in 16 bits the maximum integer one can encode is 2^16 == 65535. DataInputStream.readUTF has the exact same restriction, because it first reads the number of UTF-8 bytes to consume, in the form of a 2-byte integer, which again can only have a maximum value of 65535.

writeUTF first writes two bytes with the length, which has the same result as calling writeShort with the length and then writing the UTF-encoded bytes. writeUTF doesn't actually call writeShort - it builds up a single byte[] with both the 2-byte length and the UTF bytes. But that is why the Javadoc says "as if by the writeShort method" rather than just "by the writeShort method".

This is actually wrong statement in JavaDoc: there is no any `writeShort` analogue in `writeUTF` method. — Andremoniy, Mar 30 '14 at 07:53

DataOutputStream: purpose of the "encoded string too long" restriction

1 Answers1