2

I want to create JSON string from object.

ObjectMapper om = new ObjectMapper();
String str = om.writeValueAsString(obj);

Some objects are large, and it takes long time to create JSON string. To create 8MB JSON string, it needs about 15secs.

How can I improve this?

Alexey Gavrilov
  • 10,593
  • 2
  • 38
  • 48
N.F.
  • 3,844
  • 3
  • 22
  • 53
  • 1
    Upgrade your CPU. That's about it. Or try different 3rd party libraries for this task. Maybe one is faster for you – Kon Apr 24 '15 at 00:27
  • I think 15sec for object whitch takes 8MB to create, 15second is good enough – libik Apr 24 '15 at 00:28
  • How are you measuring the performance? (I mean, are you doing the test just once or many times and taking an average? With different datasets or just one?). You do not have to create a new `ObjectMapper` on each run if this is the case, see http://stackoverflow.com/questions/3907929/should-i-make-jacksons-objectmapper-as-static-final – BretC Apr 24 '15 at 00:30
  • 1
    That is extremely slow, even for Jackson. Something smells wrong. Are you sure your system health is OK (not in swap, out of memory, thrashing on disk or doing other things?) Could you give us a simple benchmarking example code? – BadZen Apr 24 '15 at 00:40
  • I inserted `System.nanotime()` just before/after `om.writeValueAsString(obj);`. I measured several time for a dataset. – N.F. Apr 24 '15 at 00:41
  • Also - are these POJO or in some way "magical" objects (for example, I hope you are not getting an entity object from a JPA session and feeding it to Jackson...) Let's see the actual code. – BadZen Apr 24 '15 at 00:43
  • 2
    Or - show us profiler results so we can see where the time is actually being spent. – BadZen Apr 24 '15 at 00:43
  • @BadZen I'm running my program on Google App Engine F2 instance. – N.F. Apr 24 '15 at 00:44
  • Are you creating a new ObjectMapper each time or re-using a single, shared instance (you should be doing the latter)? – jtahlborn Apr 24 '15 at 01:13

1 Answers1

5

Make sure you have enough memory: Java String for storing 8 MB of serialized JSON needs about 16 megabytes of contiguous memory in heap.

But more importantly: why are you creating a java.lang.String in memory? What possible use is there for such a huge String?

If you need to write JSON content to a file, there are different methods for that; similarly for writing to a network socket. At very least you could write output as a byte[] (takes 50% less memory), but in most cases incremental writing to an external stream requires very little memory.

15 seconds is definitely very slow. Without GC problems, after initial warmup, Jackson should write 8 megs in fraction of a second, something like 10-20 milliseconds for simple object consisting of standard Java types.

EDIT:

Just realized that during construction of the result String, temporary memory usage will be doubled as well, since buffered content is not yet cleared when String is constructed. So 8 MB would need at least 32 MB to construct String. With default heap of 64 MB this would not work well.

StaxMan
  • 113,358
  • 34
  • 211
  • 239
  • I modified my program to use `om.writeValueAsBytes(obj);` but still needs about 7.3secs (inserted `System.nanotime()` just before/after `om.writeValueAsBytes(obj)` , and difference is about 7300000000) to create 7MB JSON String. – N.F. Apr 29 '15 at 06:16
  • Ok, couple of things: (1) how much heap, and (2) is this right after starting JVM, first run, or after doing more writes? First run is slower due JVM startup, bytecode optimization and such. And then also (3) what kind of Objects do you have? – StaxMan Apr 29 '15 at 18:37
  • (1) `Runtime.getRuntime().freeMemory()` just before `writeValueAsBytes` is about 3.4M, and just after that is about 31M (larger than before), (2) it's after some write, (3) List of Objects. List size is about 5000, each object has 40 string variables. – N.F. Apr 30 '15 at 00:30
  • This would suggest that you really need to allocate more memory, since with only 3 megs left, I am guessing JVM spends lots of time on garbage-collection just because work space is so small. If you can increase heap by just something like 20 megs, that should help significantly. If you want, you can enable GC debug options to see exactly how badly JVM is working to find some more memory. :) – StaxMan Apr 30 '15 at 22:57
  • @StaxMan I came across another OutOfMemoryError. I have some questions. 1. Why does Java String take twice the continuous memory to store serialized JSON? 2. Why does byte array take half memory compared to String? Could you give me some links so that I could learn this a bit? Thanks. – wei Aug 28 '20 at 12:43
  • @wei Because most content characters in UTF-8 encoding use only 1 byte (except for east-Asian "big five", for which is usually 2...), whereas JVM internally uses `char` for characters, which is 16-bit (2 byte) datatype. Java language specification + UTF-8 description (https://en.wikipedia.org/wiki/UTF-8) should explain it. – StaxMan Aug 29 '20 at 18:26