Using MessagePack with Android

Question

Has someone tried to use MessagePack with an Android app?
Is it possible? I have tried to use the Jar from msgpack-java and received the following Exception:

Caused by: java.lang.ExceptionInInitializerError
  at org.msgpack.Packer.pack(Packer.java:532)
  at org.msgpack.MessagePack.pack(MessagePack.java:31)
  ... 15 more
  Caused by: java.lang.ExceptionInInitializerError
  at org.msgpack.template.TemplateRegistry.<clinit>(TemplateRegistry.java:38)
  ... 17 more
  Caused by: java.lang.VerifyError: org.msgpack.template.BeansFieldEntryReader
  at org.msgpack.template.builder.BeansTemplateBuilder.<init (BeansTemplateBuilder.java:42)
  at org.msgpack.template.builder.BuilderSelectorRegistry.initForJava(BuilderSelectorRegistry.java:73)
  at org.msgpack.template.builder.BuilderSelectorRegistry.<clinit>(BuilderSelectorRegistry.java:38)
  ... 18 more

The code that I use is very simple

PrintWriter out = new PrintWriter(socket.getOutputStream());
Message msg = new Message();
msg.body = "asdasdasd";
msg.from = "qwe";
msg.to = "ttt";
byte[] bytes = MessagePack.pack(msg);
out.print(bytes);
out.flush();

I have javassist.jar, msgpack-0.5.2.jar, slf4j-api-1.6.2.jar and slf4j-jdk14-1.6.2.jar in my lib directory.

In my server application this code works fine with the same libraries.

Answer 1

(Hopefully) FINAL UPDATE

msgpack : 0.6.8 works on Android without any problems

msgpack-rpc : 0.7.0 works on Android with one caveat.

Specifically, you need to add the following to onCreate for API Level 8 (Android 2.2.1), and possibly lower:

java.lang.System.setProperty("java.net.preferIPv4Stack", "true");
java.lang.System.setProperty("java.net.preferIPv6Addresses", "false");

due to this bug.

If you want to see a simple example, here's a pair of projects set up for this purpose:

• https://github.com/mikkoz/msgpack-android-test-server/tree/master/msgpack-android-test-server
• https://github.com/mikkoz/msgpack-android-test-client/tree/master/msgpack-android-test-client

---

Previous Versions

UPDATE: as of 0.6.7 msgpack should be compatible with Android (there is a small dependency exclusion issue). Check the text below for msgpack-rpc (which also might be adapted in the future).

NOTE: If you're also using msgpack-rpc, you need to do the following steps:

1. Download the msgpack-rpc source from git://github.com/msgpack/msgpack-rpc.git (specifically, the "java" folder).
2. Change the main msgpack artifact version to the one you've built.
3. In org.msgpack.rpc.loop.netty.NettyEventLoop, change the NioClientSocketChannelFactory to OioClientSocketChannelFactory(getWorkerExecutor()).
4. Build the MessagePack-RPC in the same way as in the case of the main MessagePack JAR (see Step 11 above).

The NettyEventLoop replacement is due to this issue: http://markmail.org/message/ypa3nrr64kzsyfsa .

Important: I've only tested synchronous communication. Asynchronous might not work.

---

And here's the reason for msgpack not working with Android prior to 0.6.7:

The reason for the error is that MessagePack uses several java.beans classes that are not included in the Android SDK. You're probably using the MessagePackBeans annotation.

This is a similar problem to the one described here, for which the general solution is outlined here. Unfortunately, in our case it requires a rebuild of msgpack. Here's what I did (you can almost certainly skip Steps 5 and 8, but I haven't tried it that way) :

1. Download the MessagePack source from https://github.com/msgpack/msgpack-java.git.
2. Import the MessagePack source as a project in your IDE.
3. Download the Apache Harmony source for the relevant packages from http://svn.apache.org/repos/asf/harmony/enhanced/java/trunk/classlib/modules/beans/src/main/java .
4. Copy these packages into your MessagePack project's src/main/java folder:
   • java.beans
   • java.beans.beancontext
   • org.apache.harmony.beans
   • org.apache.harmony.beans.internal.nls
5. In your MessagePack project, remove the following classes:
   • PropertyChangeListener
   • IndexedPropertyChangeEvent
   • PropertyChangeEvent
   • PropertyChangeListenerProxy
   • PropertyChangeSupport
6. Rename the java.beans packages to something different, e.g. custom.beans .
7. Change all java.beans references to the renamed ID, so again e.g. custom.beans. This applies especially to BeansFieldEntryReader (this class is the reason for the original error).
8. Change the custom.beans references for the five classes you removed in Step 5 back to java.beans.
9. In the org.apache.harmony.beans.internal.nls.Messages class, comment out the method setLocale, and remove the imports associated with it.
10. Remove all classes that still have errors, except Encoder. In that class, comment out all references to the classes you've removed. You should now have an error-free project.
11. Build the MessagePack JAR:
    • If you're using Maven, change the version in the pom.xml to something unique, run Maven build with the install goal, then add the dependency in your Android project with that version.
    • If you're not using Maven, you have to run the jar goal for Ant with the included build.xml. Replace the msgpack JAR in your Android project with this one.
12. If you're publishing your app, remember to include the relevant legal notice for Apache Harmony. It's an Apache License, just like MessagePack.

That should do it. Using your example code, and my own data class, I was successfully able to pack and unpack data.

The entire renaming ritual is due to the fact that the DEX compiler complains about java.* package naming.

Answer 2

There is an ongoing effort by @TheTerribleSwiftTomato and the MessagePack core team to get MessagePack working on Android, please see the related GitHub issue. The fix mentioned in @TheTerribleSwiftTomato's answer is to be found here.

Update

I've managed to get it at least running on Android by (painstakingly) adding all the necessary javassist Classes which are currently required for the build to succeed. An extra 600KB gain in size, yet at least it seems to work. All in all, it appears to be working to some extent on Android, eventually check out the lesser-known resources about Message Pack such as its User Group and its Wiki for more information.

On a side-note, be sure to use a HTTP Request Library (such as LoopJ's Android Async HTTP or Apache's HttpClient) which can handle binary data.

Last but not least you can ping me if there is interest in this jar which makes MessagePack seemingly work on Android – credits go out of course to @TheTerribleSwiftTomato who supplied the fix above!

Answer 3

I suggest you write this in the main proguard-rules file-

-dontwarn org.msgpack.**
-keep class org.msgpack.** { *; }

Answer 4

There is a critical msgpack bug saying data packed with msgpack will get corrupted on the Dalvik VM. http://jira.msgpack.org/browse/MSGPACK-51