Java: object to byte[] and byte[] to object converter (for Tokyo Cabinet)

Question

I need to convert objects to a byte[] to be stored in the Tokyo Cabinet key-value store. I also need to unbyte the byte[] to an Object when reading from the key-value store.

Are there any packages out there that will help me with this task? Or would the best solution to implement it myself?

Make your class [Serializable](http://download.oracle.com/javase/6/docs/api/java/io/Serializable.html), then use the answer at [ Java Serializable Object to Byte Array ](http://stackoverflow.com/questions/2836646/java-serializable-object-to-byte-array). — Matthew Flaschen, Sep 17 '10 at 14:14
If the object is not Serializable but [Parcealable](https://developer.android.com/reference/android/os/Parcelable.html) or a [Parcel](https://developer.android.com/reference/android/os/Parcel), you can use [marshall](https://developer.android.com/reference/android/os/Parcel#marshall()) and [unmarshall](https://developer.android.com/reference/android/os/Parcel#unmarshall(byte%5B%5D,%20int,%20int)) methods, see (https://stackoverflow.com/a/18000094/9160102) — Bruno L., Sep 29 '19 at 07:12

Thomas Mueller · Accepted Answer · 2013-05-30T06:58:22.483

174

public static byte[] serialize(Object obj) throws IOException {
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    ObjectOutputStream os = new ObjectOutputStream(out);
    os.writeObject(obj);
    return out.toByteArray();
}
public static Object deserialize(byte[] data) throws IOException, ClassNotFoundException {
    ByteArrayInputStream in = new ByteArrayInputStream(data);
    ObjectInputStream is = new ObjectInputStream(in);
    return is.readObject();
}

edited May 30 '13 at 06:58

answered Sep 17 '10 at 14:33

Thomas Mueller

48,905
14
116
132

don't forget to call `is.close` and `in.close`. – Cacho Santa May 29 '13 at 23:27
21

`is.close` and `in.close` are not needed in this case because it's an in-memory stream. The only thing it would do here is make it slower and more verbose. But I understand the IDE might flag the code above because the stream is not closed. – Thomas Mueller May 30 '13 at 06:56
3

I don't think it will make slower, it is only an empty method so the time difference must be very low. But you are right: `Closing a ByteArrayOutputStream has no effect.`[Java Doc](http://docs.oracle.com/javase/6/docs/api/java/io/ByteArrayOutputStream.html#close%28%29) Awesome :D – Cacho Santa May 30 '13 at 16:32
What if the object to be serialized is of type InputStream? – Arthur Eirich Jan 05 '15 at 11:02
@ArthurEirich Generally it would still work, but it might not do what you expect. – Thomas Mueller Jan 05 '15 at 14:12
1

@ThomasMueller Unfortnately, "os.writeObject(obj);" would cause a NotSerializableException as InputStream does not implement the Serializable interface =( – Arthur Eirich Jan 05 '15 at 14:20
@ArthurEirich OK, it would only work if the InputStream implementation implements Serializable. – Thomas Mueller Jan 05 '15 at 14:41
@all if the object is Parcelable then what is solution ?? – Zala Janaksinh Jun 15 '15 at 09:42
@ZalaJanaksinh that's a different question. This question is about Java, not Android. – Thomas Mueller Jun 15 '15 at 12:03
are there any memory limitations using this method? – Jenya Kirmiza Apr 04 '17 at 07:54
Yes, those operations can run out of memory, specially deserialize: it's relatively easy to construct a (small) byte array that will be deserialized into a extremely large object, which will cause out of memory. There is also a security risk to deserialize any byte array, but this answer is not about security aspects. – Thomas Mueller Apr 04 '17 at 12:18
this method is causing "IOException: java.io.StreamCorruptedException: invalid stream header: 7B227374", any idea why its doing so? – Safeer Oct 05 '17 at 06:54
@Safeer probably because you tried to deserialize, but the input (the byte array) contains a JSON text file starting with `{"st...`. That won't work. You can only deserialize things that were serialized with the above method. To read a JSON file, try searching for "convert JSON to Java object" or similar. – Thomas Mueller Oct 05 '17 at 09:37
code not work for offheap cache like ohc. when performing benchmark, the code will throw nullpointer exception. – CharlieShi Aug 16 '18 at 03:30
@CharlieShi I think you are not using it correctly. Please ask a new question. – Thomas Mueller Aug 16 '18 at 07:01

score 14 · Answer 2 · answered Sep 17 '10 at 14:14

14

If your class extends Serializable, you can write and read objects through a ByteArrayOutputStream, that's what I usually do.

answered Sep 17 '10 at 14:14

G B

2,951
2
28
50

3

All types included as variables in your class (and all types in those types, and so on) also need to be Serializable. – Dean J Sep 17 '10 at 14:19
Actually he'll need an ObjectOutputStream to wrap the BAOS too... But yeah, that's the easiest way to go. – Aviad Ben Dov Sep 17 '10 at 14:23
I'd consider using something other than Java's built-in serialization, too - JBoss serialization, JSerial, Avro, etc, or an XML format like XStream or Javolution marshalling plus gzip. Standard serialization is not particularly fast, and although its marginal space efficiency is good, there's quite a bit of per-stream overhead. – Tom Anderson Sep 17 '10 at 16:28

score 11 · Answer 3 · answered Mar 31 '14 at 05:16

11

Use serialize and deserialize methods in SerializationUtils from commons-lang.

answered Mar 31 '14 at 05:16

SANN3

9,459
6
61
97

score 6 · Answer 4 · answered Dec 23 '20 at 12:28

6

You can use ObjectMapper

        ObjectMapper objectMapper = new ObjectMapper();
        ObjectClass object = objectMapper.readValue(data, ObjectClass.class);

answered Dec 23 '20 at 12:28

S. Du

711
7
10

score 5 · Answer 5 · edited Dec 23 '20 at 03:51

5

You can look at how Hector does this for Cassandra, where the goal is the same - convert everything to and from byte[] in order to store/retrieve from a NoSQL database - see here. For the primitive types (+String), there are special Serializers, otherwise there is the generic ObjectSerializer (expecting Serializable, and using ObjectOutputStream). You can, of course, use only it for everything, but there might be redundant meta-data in the serialized form.

I guess you can copy the entire package and make use of it.

edited Dec 23 '20 at 03:51

Christian Wilkie

3,693
5
34
49

answered Sep 17 '10 at 14:14

Bozho

588,226
146
1,060
1,140

link is broken! – tgabb Mar 03 '20 at 23:27

score 0 · Answer 6 · answered Jun 24 '21 at 07:40

If you do not want to serialize you can use object mapper

ObjectMapper objectMapper = new ObjectMapper();
objectMapper.registerModule(new JavaTimeModule());

ByteArrayOutputStream out = new ByteArrayOutputStream();
objectMapper.writeValue(out,obj);
byte[] data = out.toByteArray();

When should we implement Serializable interface?

Java: object to byte[] and byte[] to object converter (for Tokyo Cabinet)

6 Answers6

Linked