Same thread running the same recursive code seems to consume more stack memory in Java 8 compared to Java 7

Question

I am asking a question about "java stack overflow" in the "stackoverflow" site :)

A particular thread which makes some recursive function calls for a particular input runs fine in Oracle Java 7 (64 bit) for a configured stack size of 228k (-Xss228k).

However, the same thread running the same recursive code for the same input throws a java.lang.StackOverflowError in Oracle Java 8 (64 bit) for the same stack size of 228k. It runs fine in Java 8 if the stack size is increased to 512k (-Xss512k).

Any idea why this could happen? Have any changes been made in Java 8 (Hotspot JVM) compared to Java 7 which could increase the stack memory consumption for recursive function calls? I can provide additional details if required.

(Edit) NOTE: The same recursion depth works "always" in Java 7 but fails "always" in Java 8 for a stack size of 228k.

Stack consumption is not exactly predictable. Without code examples, it’s impossible to tell whether one out of the dozens of possible reasons applies to your particular issue. See for example [“Why is the max recursion depth I can reach non-deterministic?”](http://stackoverflow.com/q/27043922/2711488) — Holger, Sep 11 '15 at 10:34
@Holger The same recursion depth works "always" in Java 7 but fails "always" in Java 8 for a stack size of 228k. So, do the non-deterministic factors still apply in my scenario? Meanwhile, I will try to come up with a code example soon. — Sanjay Bhat, Sep 11 '15 at 10:51
Are your JVMs both 64bit? Both 32bit? One is 32, other is 64? — Tagir Valeev, Sep 11 '15 at 11:47

score 5 · Answer 1 · answered Sep 11 '15 at 12:22

I wrote a small test for different recursion scenarios (static or instance method, different number of int parameters). Here's the results (number of calls before StackOverflowError occurs) on different versions of HotSpot JVM 64bit with -Xss228k option. Note that numbers differ somewhat between runs (I launched twice with every JVM):

          St/0  St/1  St/2  St/3  St/4  In/0  In/1  In/2  In/3  In/4
1.7.0_60  2720  2519  2309  2131  1979  2519  2309  2131  1979  1847
1.7.0_60  2716  2516  2306  2128  1976  2516  2306  2128  1976  1845
1.7.0_79  2716  2516  2306  2128  1976  2516  2306  2128  1976  1845
1.7.0_79  2729  2528  2317  2139  1986  2528  2317  2139  1986  1853
1.7.0_80  2718  2518  2308  2130  1978  2518  2308  2130  1978  1846
1.7.0_80  2738  2536  2324  2146  1992  2536  2324  2146  1992  1859
____________________________________________________________________
1.8.0_25  2818  2469  2263  2089  1940  2469  2263  2089  1940  1810
1.8.0_25  3279  2468  2262  2088  1939  2468  2262  2088  1939  1810
1.8.0_40  2714  2467  2262  2088  1938  2467  2262  2088  1938  1809
1.8.0_40  2735  2486  2279  2104  1953  2486  2279  2104  1953  1823
1.8.0_60  2729  2481  2274  2099  1949  2481  2274  2099  1949  1819
1.8.0_60  2719  2472  2266  2091  1942  2472  2266  2091  1942  1812
____________________________________________________________________
1.9_b80   2717  2470  2264  2090  1941  2470  2264  2090  1941  1811
1.9_b80   2715  2468  2263  2088  1939  2468  2263  2088  1939  1810

It's quite expected that Instance/0 is the same as Static/1 and so on as instance call need to pass this as an additional argument.

So indeed there's some degradation of number of recursive calls allowed (except for Static/0 case) betwee JDK 7 and JDK 8: you lose around 30-40 calls (roughly 5%) of total count. So probably in your application you was very close to the limit. By the way I noticed a sudden jump between -Xss256k and -Xss260k (tested on 1.8.0_40):

          St/0  St/1  St/2  St/3  St/4  In/0  In/1  In/2  In/3  In/4
-Xss256k  2724  2476  2270  2095  1945  2476  2270  2095  1945  1816
-Xss260k  4493  3228  2959  2731  2536  3228  2959  2731  2536  2367

So you may try to increase stack size to -Xss260k and it should be enough for your task.

By the way 32-bit JVM allows much more calls with the same -Xss228k:

          St/0  St/1  St/2  St/3  St/4  In/0  In/1  In/2  In/3  In/4
7u67_32b  7088  5078  4655  4297  3990  5078  4655  4297  3990  3724
7u67_32b  6837  5092  4667  4308  4001  5092  4667  4308  4001  3734

Thus it's also possible that you've switched from 32-bit Java-7 to 64-bit Java-8. In this case, of course much more stack space is needed as even with compressed OOPs pointers in the stack seem to be 64bit, thus occupying more space.

So your test didn’t give the JIT a chance to kick in? As shown in the linked question, it can change the results by factor six… — Holger, Sep 11 '15 at 17:27
@Holger, for some reason St/0 is the less stable case (see 1.8.0_25/St_0) and it was tested in linked question. For the JIT-compiled code the recursion level can be much higher (up to 13000 calls in the first test). The question is whether the compiler has enough time to JIT-compile. Usually the program is dead on the first StackOverflow error, thus I omitted the analysis of the further launches. In general my results are consistent with yours. Probably in St/0 JIT-compiler has some chance. And of course the results might differ with more complex method body (backedges, etc.) — Tagir Valeev, Sep 12 '15 at 01:42
@TagirValeev Thanks a lot for the detailed answer! In my case, both JVMs are 64 bit. As you mentioned, this seems to be the case in my scenario: "So indeed there's some degradation of number of recursive calls allowed (except for Static/0 case) betwee JDK 7 and JDK 8: you lose around 30-40 calls (roughly 5%) of total count. So probably in your application you was very close to the limit. By the way I noticed a sudden jump between -Xss256k and -Xss260k. So you may try to increase stack size to -Xss260k and it should be enough for your task." — Sanjay Bhat, Sep 13 '15 at 15:15
@TagirValeev I was also going to try and come up with a stack size between 228K and 512k which would suffice for my thread. I will most likely end up setting the stack size to 260k as you have suggested :) By the way, do you have any idea as to why we are observing the 5% degradation (roughly) in the number of recursive calls allowed between Java 7 and Java 8? Any Java 8 documentation which talks about changes which can lead to such a degradation? — Sanjay Bhat, Sep 13 '15 at 15:28
@TagirValeev FYI for my scenario, I found that the thread which worked fine for a stack size of 228k in Java 7 needed a stack size of 328k in Java 8 i.e., 100k more than Java 7. — Sanjay Bhat, Sep 15 '15 at 10:08
JDK 7 has 2 yellow pages on thread stack, but JDK 8 has 3 yellow pages on Windows, hence the difference. Extra stack page (4K) provides right about 50 more interpreted calls. [Here is](http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2013-June/007679.html) the relevant discussion on *hotspot-runtime-dev* list, and [here is](http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/rev/cd54c7e92908) the corresponding fix in JDK 8. — apangin, May 23 '16 at 10:40
Another related difference between JDK 7 vs. JDK 8 is JIT compilation policy. E.g. JDK 8 has TieredCompilation on by default. This means Java methods are first compiled with C1, which is more greedy in stack usage comparing to C2. See the [related question](http://stackoverflow.com/questions/35517934/why-does-the-count-of-calls-of-a-recursive-method-causing-a-stackoverflowerror-v). — apangin, May 23 '16 at 10:53

Same thread running the same recursive code seems to consume more stack memory in Java 8 compared to Java 7

1 Answers1

Linked

Related