How is ArrayOutOfBoundsException possible in String.valueOf(int)?

Question

Why does this code sometimes produce ArrayOutOfBoundsException? How is that even possible for String.valueOf(int)?

public static String ipToString(ByteString bs) {
  if (bs == null || bs.isEmpty()) {
    return null;
  } else {
    StringBuilder sb = new StringBuilder();
    boolean started = false;
    for (Byte byt : bs) {
      if (started) {
        sb.append(".");
      }
      sb.append(String.valueOf(byt & 0xFF));
      started = true;
    }

    return sb.toString();
  }
}


java.lang.ArrayIndexOutOfBoundsException: -81914
  at java.lang.Integer.getChars(Integer.java:458)
  at java.lang.Integer.toString(Integer.java:402)
  at java.lang.String.valueOf(String.java:3086)
  at com.mystuff.mypackage.ipToString(MyCode.java:1325)
  ...
  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)

Updates

I don't know the value of the byte when this occurs, but it doesn't seem like it should be possible for any possible value of byte.

Once it happens once, every invocation then errors out with the same exception.

Environment:

java version "1.8.0_20"
Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)

Does this fall over outside of a multithreaded environment? The stack trace suggests you're running this as part of a thread pool. Is there a synchronization problem? — chiastic-security, Sep 19 '14 at 20:42
@chiastic-security Whatever you do with multithreading, a pure function like `Integer.toString` must not fail. — Marko Topolnik, Sep 19 '14 at 20:44
In case you are using IBM JVM, there seems to be a bug in the JIT compiler.. http://stackoverflow.com/questions/16059794/stringbuilder-arrayindexoutofboundsexception — Naresh Vavilala, Sep 19 '14 at 20:45
@MarkoTopolnik Well, yes, indeed. Quite right. But I think I would still check whether it fails in as simple a context as possible. The point is that `Integer.toString` is failing somehow! — chiastic-security, Sep 19 '14 at 20:46
@chiastic-security Unsafe publishing would be the best bet, but a simple `int` would have to be resilient even to that. — Marko Topolnik, Sep 19 '14 at 20:46
@NareshVavilala Unrelated, this exception does not occur in `StringBuilder` code. — Marko Topolnik, Sep 19 '14 at 20:47
@JeanLogeart I can exactly match the line numbers to my JDK 8 source code. — Marko Topolnik, Sep 19 '14 at 20:48
@MarkoTopolnik I figured it out since it was not working with 7 :o — Jean Logeart, Sep 19 '14 at 20:51
java version "1.8.0_20" Java(TM) SE Runtime Environment (build 1.8.0_20-b26) Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode) — ʞɔıu, Sep 19 '14 at 20:51
@HotLicks It's part of Google's Protocol Buffers. It's basically just an immutable list of bytes. — chiastic-security, Sep 19 '14 at 20:52
@HotLicks No, Protocol Buffers is a general Java implementation, not restricted to Android. — chiastic-security, Sep 19 '14 at 20:54
It does appear to be more consistent with a JITC bug than anything else. — Hot Licks, Sep 19 '14 at 20:56
Really, the only way to get to the bottom of this is to find out what the value of `byt` is. I suggest echoing `byt & 0xFF` to the console just before that line, so that you can catch the last one before it blows up. Then try a minimal program that just invokes `String.valueOf()` on that byte value, and see what happens. — chiastic-security, Sep 19 '14 at 21:02
@chiastic-security Chances of reproducing like that are one in a million. — Marko Topolnik, Sep 19 '14 at 21:04
You will probably find byt & 0xFF is an integer; you might try byt & ((byte)0xFF) or add another (byte) in front of it. It should not cause this error though — Bruce Martin, Sep 19 '14 at 21:04
@chiastic-security Chances are that it has nothing to do with the actual value of that `byte`: the code is bulletproof, the only thing that could trigger that issue would be a bug in JVM. — Sergey Kalinichenko, Sep 19 '14 at 21:05
@dasblinkenlight Sure, but you'd want to find a minimal (non-)working example, and that means reducing the code to the smallest point where you can reliably trigger the bug. — chiastic-security, Sep 19 '14 at 21:07
@chiastic-security There's no reliable triggering of this bug... I've just written a `jmh` benchmark utilizing all 4 cores to do work like in the question, going through all `byte` values, no exceptions. — Marko Topolnik, Sep 19 '14 at 21:08
@BruceMartin `byt & 0xFF` is intentional. It is the standard way to interpret the byte as an unsigned 8-bit integer. — Marko Topolnik, Sep 19 '14 at 21:12
it's intermittent. There are only 255 possible byte values and this code is executed at high volume of requests on a server serving production traffic — ʞɔıu, Sep 19 '14 at 21:16
When this error occurs, is the index out of bounds always `-81914` or does that number change? — durron597, Sep 19 '14 at 21:18
always that value, and once it happens once every subsequent invocation bombs in the exact same way — ʞɔıu, Sep 19 '14 at 21:18
Your last update is important: it quite cleanly points to JIT compiler issue (inlining or possibly even some assembler macro code). — Marko Topolnik, Sep 19 '14 at 21:19
will try -Xjit:exclude={ProgramClass.callStringGetChars*} when I get the chance — ʞɔıu, Sep 19 '14 at 21:27
-Xjit:exclude={ProgramClass.callStringGetChars*} isn't accepted... — ʞɔıu, Sep 19 '14 at 21:56

apangin · Accepted Answer · 2014-09-21T11:54:45.047

19

This is a JIT compiler bug that has been introduced in JDK 8u20 as a side-effect of another fix:
JDK-8042786

The problem is related to auto-boxing elimination optimization.
The work-around is to switch the optimization off by -XX:-EliminateAutoBox JVM flag

Looks like the problem also exists in the most recent JDK 9 source base.
I've submitted the bug report: https://bugs.openjdk.java.net/browse/JDK-8058847 with 100% reproducible minimal test case included.

edited Sep 21 '14 at 11:54

answered Sep 20 '14 at 22:21

apangin

92,924
10
193
247

2

a patch has been accepted to resolve the issue http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/7723d5b0fca3 – ʞɔıu Oct 24 '14 at 21:41

Marko Topolnik · Answer 2 · 2014-09-22T13:20:50.253

I can reliably reproduce your issue with this code:

public class Main
{
  public static StringBuilder intToString(byte[] bs) {
    final StringBuilder sb = new StringBuilder();
    boolean started = false;
    for (Byte byt : bs) {
      if (started) sb.append(".");
      sb.append(String.valueOf(byt & 0xFF));
      started = true;
    }
    return sb;
  }

  public static void main(String[] args) {
    final byte[] bs = {-2, -1, 0, 1, 2};
    while (true) intToString(bs);
  }
}

The issue will almost certainly be traced to a JIT compiler bug. Your observation that, once it happens the first time, it happens reliably on every subsequent call, points cleanly to a JIT compilation event which introduces the buggy code into the codepath.

If that's available to you, you could activate diagnostic JVM options which will print all compilation events (-XX:PrintCompilation). Then you may be able to correlate such an event with the moment when the exception starts appearing.

if the problem is caused by `for(Byte b : bs)` as opposed to `for(byte b : bs)` - see comments in my answer - can you see if you can reproduce the problem with that change? — durron597, Sep 22 '14 at 12:50
See my update: that change makes it easily reproducible. One thing that wasn't obvious from your code was whether `bs` returned a `byte` or a `Byte`, and that turns out to be a key difference. — Marko Topolnik, Sep 22 '14 at 13:21

score 7 · Answer 3 · edited Sep 22 '14 at 20:45

7

I am leaving the code snippet here, as it still ought to be run faster than the original code - at a cost of memory - but be advised it doesn't actually fix the problem.

private static final String[] STRING_CACHE = new String[256];

static {
  for(int i = 0; i <= 255; i++) {
    STRING_CACHE[i] = String.valueOf(i);
  }
}

public static String ipToString(ByteString bs) {
  if (bs == null || bs.isEmpty()) {
    return null;
  } else {
    StringBuilder sb = new StringBuilder();
    boolean started = false;
    for (Byte byt : bs) {
      if (started) {
        sb.append(".");
      }
      sb.append(STRING_CACHE[byt & 0xFF]);
      started = true;
    }

    return sb.toString();
  }
}

edited Sep 22 '14 at 20:45

ʞɔıu

47,148
35
106
149

answered Sep 19 '14 at 21:28

durron597

31,968
17
99
158

1

certainly is not enough and workaround is also not asked by OP. – Kumar Abhinav Sep 19 '14 at 21:33
8

@KumarAbhinav From the help center [How to answer](http://stackoverflow.com/help/how-to-answer): "What, specifically, is the question asking for? Make sure your answer provides that – or **a viable alternative**" – durron597 Sep 19 '14 at 21:37
3

It should be noted that this bug is described elsewhere as having been fixed a year ago. The OP needs to update his JVM version. – Hot Licks Sep 20 '14 at 02:10
1

@HotLicks But OP is clearly using the newest JVM available? Still, a link to the bug would be the closest to a real answer to this question! – Marko Topolnik Sep 20 '14 at 06:29
@MarkoTopolnik - Well, I guess the fixed version (to a nearly identical bug) was in a IBM JVM, whereas this is apparently an Oracle version, so perhaps it's not the same. (Or perhaps it is the same but Oracle hasn't fixed it yet.) – Hot Licks Sep 20 '14 at 12:38
I've verified that your suggested workaround crashes JVM - even worse than ArrayIndexOutOfBoundsException. – apangin Sep 22 '14 at 07:17
See https://bugs.openjdk.java.net/browse/JDK-8058847. The bug is not in `String.valueOf` but rather in autoboxing optimization. If replace `for (Byte byt : bs)` with `for (byte byt : bs)` the problem will disappear. – apangin Sep 22 '14 at 12:37
This can be reproduced only on JDK 8u20 and JDK 9. Earlier versions work fine since they have `EliminateAutoBox` disabled by default. – apangin Sep 22 '14 at 12:41

How is ArrayOutOfBoundsException possible in String.valueOf(int)?

3 Answers3

Linked