0

While decoding an Base64 encoded string to byte array (Have to do this as I have a key which can act on byte array to decrypt), I am getting outOfMemory. What are the effective ways to handle this problem? Should I chunk my input encoded String into partitions of size and then decode it or any other suggestions which are effective please suggest.

Code which was causing the issue.

byte[] encrypted = Base64.decodeBase64(strEncryptedEncodedData);

Stack Trace

DefaultQuartzScheduler_Worker-3
  at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:48)
  at java.lang.StringCoding$StringEncoder.encode([CII)[B (StringCoding.java:300)
  at java.lang.StringCoding.encode(Ljava/lang/String;[CII)[B (StringCoding.java:344)
  at java.lang.String.getBytes(Ljava/lang/String;)[B (String.java:918)
  at org.apache.commons.codec.binary.StringUtils.getBytesUnchecked(Ljava/lang/String;Ljava/lang/String;)[B (StringUtils.java:156)
  at org.apache.commons.codec.binary.StringUtils.getBytesUtf8(Ljava/lang/String;)[B (StringUtils.java:129)
  at org.apache.commons.codec.binary.BaseNCodec.decode(Ljava/lang/String;)[B (BaseNCodec.java:306)
  at org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B (Base64.java:669)

Eclipse Memory Analyzer memory usage:

enter image description here

Edit1: Max allowed XMX is 1 GB.

Edit2: JDK version"1.8.0_91"

Prashanth
  • 107
  • 2
  • 12

2 Answers2

0

try to increase max heap size to the JVM using option like this -Xmx4096m

Don Ha
  • 689
  • 5
  • 9
  • sorry max allowed is 1GB in our environments, I would prefer a better way of partitioning the string or any clever tricks to not load entire memory. – Prashanth Jul 06 '20 at 19:11
0

Please specify the java version you use for this code.

There are more than 10 different types of OutOfMemoryError as listed below and yours might be “10.java.lang.OutOfMemoryError: Direct buffer memory” type. Please verify your exception stack trace to find this matching string to confirm the same. If you see different type, please share it.

I verified that “java.lang.StringCoding$StringEncoder” class you shared in your exception trace uses java.nio.ByteBuffer and other related classes. You can validate the import sections in the below url. http://cr.openjdk.java.net/~sherman/7040220/webrev/src/share/classes/java/lang/StringCoding.java.html

Java applications can access native memory (not heap memory) for buffer operations (direct byte) to perform speed operations. Some portion of memory is allocated to JVM from native memory for these direct byte buffer operations. If its size is not enough, you can increase it using VM flag –XX:MaxDirectMemorySize= (eg. -XX:MaxDirectMemorySize=10M). Increasing heap memory by using –Xmx flag would not solve this type of outofmemory. Please try MaxDirectMemorySize flag and see whether it solves your problem.

If you want to know more details about this OutOfMemoryError, you can read Java Performance Optimization: How to avoid the 10 OutOfMemoryErrors book.

1.java.lang.OutOfMemoryError: Java heap space
2.java.lang.OutOfMemoryError: Unable to create new native thread
3.java.lang.OutOfMemoryError: Permgen space
4.java.lang.OutOfMemoryError: Metaspace
5.java.lang.OutOfMemoryError: GC overhead limit exceeded
6.java.lang.OutOfMemoryError: Requested array size exceeds VM limit
7.java.lang.OutOfMemoryError: request "size" bytes for "reason". Out of swap space?
8.java.lang.OutOfMemoryError: Compressed class space
9.java.lang.OutOfMemoryError: "reason" "stack trace" (Native method)
10.java.lang.OutOfMemoryError: Direct buffer memory
Nirmal
  • 89
  • 6
  • Thanks for the comment @Nirmal, We are using JDK version "1.8.0_91", from another post I am reading (https://stackoverflow.com/questions/3773775/default-for-xxmaxdirectmemorysize) I see a reply saying -XX:MaxDirectMemorySize will be set to xmx if not set directly, We are not setting MaxDirectMemorySize but we have very less -XX:MaxMetaspaceSize=256M . Will try increasing MetaSpaceSie to 1g and see if that works. – Prashanth Jul 13 '20 at 01:01