How Do I Code Large Static Byte Arrays In Java?

Question

I'm writing a code generator that is replaying events recorded during a packet capture.

The JVM is pretty limited - it turns out. Methods can't be >64KB in size. So I added all kinds of trickery to make my code generator split up Java methods.

But now I have a new problem. I was taking a number of byte[] arrays and making them static variables in my class, e.g.:

public class myclass {
    private static byte[] byteArray = { 0x3c, 0x3f, ...
        ...
    };
    private static byte[] byteArray2 = { 0x1a, 0x20, ...
        ...
    };

    ...

    private static byte[] byteArray_n = { 0x0a, 0x0d, ...
        ...
    };        
}

Now I get the error: "The code for the static initializer is exceeding the 65535 bytes limit".

I DO NOT WANT TO HAVE AN EXTERNAL FILE AND READ IN THE DATA FROM THERE. I WANT TO USE CODE GENERATED IN A SINGLE FILE.

What can I do? Can I declare the arrays outside the class? Or should I be using a string with unicode for the values 128-255 (e.g. \u009c instead of (byte)0x9c)? Or am I the only person in the world right now that wants to use statically initialised data?

UPDATE

The technique I'm now using is auto-creation of functions like the following:

private byte[] byteArray_6() {
  String localString = "\u00ff\u00d8\u00ff\u00e0\u0000\u0010JFIF\u0000" +
    "(0%()(\u00ff\u00db\u0000C\u0001\u0007\u0007\u0007\n\u0008\n\u0013\n" +
    "\u0000\u00b5\u0010\u0000\u0002\u0001\u0003\u0003\u0002\u0004\u0003";
  byte[] localBuff = new byte[ localString.length() ];
  for ( int localInt = 0; localInt < localString.length(); localInt++ ) {
    localBuff[localInt] = (byte)localString.charAt(localInt);
  }
  return localBuff;
}

Note: Java keeps on surprising. You'd think you could just encode every value in the range 0-255 as \u00XX (where XX is the 2-character hex representation). But you'd be wrong. The Java compiler actually thinks \u000A is a literal "\n" in your code - which breaks the compilation of your source code. So your strings can be littered with Unicode escapes but you'll have to use "\n" and "\r" instead of \u000a and \u000d respectively. And it doesn't hurt to put printable characters as they are in the strings instead of the 6 character Unicode escape representation.

Answer 1

Generally, you would put the data in a literal String and then have a method which decodes that to a byte[]. toByteArray() is of limited use as UTF-8 wont produce all possible byte sequences, and some values don't appear at all.

This technique is quite popular when trying to produce small object code. Removing huge sequences of array initialisation code will also help start up time.

Off the top of my head:

public static byte[] toBytes(String str) {
    char[] src = str.toCharArray();
    int len = src.length;
    byte[] buff = new byte[len];
    for (int i=0; i<len; ++i) {
        buff[i] = (byte)src[i];
    }
    return buff;
}

More compact schemes are available. For instance you could limit string character contents to [1, 127] (0 is encoded in a non-normalised form for really bad reasons). Or something more complicated. I believe JDK8 will have a public API for Base64 decoding which isn't too bad and nicely standardised.

Answer 2

declare an arraylist and use a static constructor

Answer 3

May by you can use nested classes for storing static arrays. This step is not the best in means of performans, but I think you could get it with minimum changes in your code.