java, writing data size with DataOutputStream

Question

I have a series of objects which write their data out to a file via DataOutputStream.

As my application file format evolves, I will be adding further data objects to it, which should be able to be read by "older" versions of the application.

To be able to do this, I am needing to preface the objects data with a size value indicating the number of bytes that the current object will take up.

However, as this data can be a variable size, notably when handling strings, I cannot know beforehand what the size will be.

Is there a way to "pre-write" to a byte buffer, as if it were a DataOutputStream (methods in particular - writeByte, writeShort, writeInt, writeUTF) which will enable me to get back the byte length of the data before writing it out to the DataOutputStream ? Doing this will enable me to skip over newer data objects that older versions of the application does understand.

Are you serializing objects yourself, or using java's built in serialization? — Socratic Phoenix, Oct 17 '16 at 19:08
Do you need to store the data in binary format, or could you marshal your data objects into text, e.g. JSON or YAML with a library like Jackson? Deserialization should also become quite simple when using a library for the heavy lifting. — Mick Mnemonic, Oct 17 '16 at 19:09
If you're serializing it yourself, I use [this](https://github.com/PlutoPowered/Nebula/blob/master/src/main/java/com/gmail/socraticphoenix/nebula/io/Bytes.java#L29) technique to serialize strings, but you could potentially use null termination as well — Socratic Phoenix, Oct 17 '16 at 19:12
I am serializing objects myself. I am NOT using any form of text serialisation. This is therefore a binary format, hence the need to be able to ascertain the data size. — Simon, Oct 18 '16 at 07:13

score 1 · Answer 1 · answered Oct 17 '16 at 19:14

Regular Java serialization is not that great for a variety of reasons. One of the most important ones is that it's very brittle, and tends not to be "future proof". If possible, I'd suggest you use a different serialization format, especially if you specifically mention that you plan on adding fields to the class you serialize.

Formats such as Protobuf, and JSON have Java libraries with nice APIs, and good forwards/backwards compatibility features. In most cases, it would be far simpler to Serialize your data into a more convenient format, than solving the problems of the existing one.

score 0 · Answer 2 · edited May 23 '17 at 12:06

As you can see here, you could use java.lang.instrument.Instrumentation in this way:

import java.lang.instrument.Instrumentation;

public class ObjectSizeFetcher {
    private static Instrumentation instrumentation;

    public static void premain(String args, Instrumentation inst) {
        instrumentation = inst;
    }

    public static long getObjectSize(Object o) {
        return instrumentation.getObjectSize(o);
    }
}

and then calculate the size in bytes of your object:

public static void main(String [] args) {
   Object myObject = ...;
        System.out.println(ObjectSizeFetcher.getObjectSize(myObject));
    }

Looks interesting. However it is not the size of the object I am looking for but the size, or length, of binary data which will be written out. — Simon, Oct 18 '16 at 07:14

java, writing data size with DataOutputStream

2 Answers2