1

We ran into a problem all of a sudden after switching our base jdk images and an upgrade of k8s, specifically we wanted to use eclipse-temurin:8u312-b07-jdk. Here is a very simplified example:

kubectl run jdk --image eclipse-temurin:8u312-b07-jdk --limits "memory=2Gi"
kubectl exec jdk -it -- /bin/bash
nano Dummy.java

with content:

import java.util.concurrent.locks.LockSupport;
import java.util.concurrent.TimeUnit;
public class Dummy {
   public static void main(String[] args) throws Exception {
       while(true) {
           LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(1));
       }
   }
}

compile and run it:

javac Dummy.java
java -XX:+UnlockExperimentalVMOptions  -XX:+UseCGroupMemoryLimitForHeap Dummy

And we get a warning:

OpenJDK 64-Bit Server VM warning: Unable to open cgroup memory limit file /sys/fs/cgroup/memory/memory.limit_in_bytes (No such file or directory)

well, yes. There is no such file, but there is cat /sys/fs/cgroup/memory.max (cgroup v2).

Does that mean that UseCGroupMemoryLimitForHeap will have no effect? It surely looks like it. I know it is deprecated and removed in 11 - that's not the point.

Thank you.

I did find this issue and it looks like I can do:

java -XX:+UnlockExperimentalVMOptions -XX:MaxRAM=$(cat /sys/fs/cgroup/memory.max)  Dummy

but that is something I do not really like.

Eugene
  • 117,005
  • 15
  • 201
  • 306
  • Cgroup v2 is supported starting from JDK 15, see [JDK-8230305](https://bugs.openjdk.java.net/browse/JDK-8230305). No backports to JDK 8 or 11 yet. – apangin Feb 16 '22 at 20:34
  • IMO, `MaxRAMPercentage` is a harmful flag, since it gives a wrong perception that the JVM adapts to the given environment. It practice, it almost never does. A belief that heap size should depend on the amount of available RAM is erroneous; most often, it works the other way round. If your application perfectly works with 1GB heap, just set `-Xmx1G` and enjoy. If the heap is larger than application requires - it's the waste of memory. If it's smaller - the application basically won't work at all. – apangin Feb 16 '22 at 20:46
  • Thank you @apangin for the bug reference. As to your point: we are in a business where once a season ( think 2/3 months ), we are hit with billions of kafka messages, very, very fast. For 1 or 2 days, I need our pods to have a very big heap. We know the day when it starts, so we just restart the pods with a much bigger limit. For our business model, taking a % of RAM, makes all the sense in the world. – Eugene Feb 17 '22 at 04:01
  • Your example only supports my point: you know that sometimes the application needs a huge heap - so why not setting the exact heap size with -Xmx? To make Xmx configurable from outside, set it via an environment variable. `MaxRAMPercentage` seems to me a wrong way. E.g. normally the application works with 1GB heap, so you can start it in a 2GB container with -Xmx1G. If in rare cases the application requires 60GB heap, start it with -Xmx60G and 64GB container limit. Otherwise `-XX:MaxRAMPercentage=50` will waste a huge amount of RAM. – apangin Feb 17 '22 at 10:44
  • @apangin I see your point, but these spikes in what we want are very costly on our provider, we we only request when needed. and btw `XX:MaxRAMPercentage=50` was just an example, we have it as `XX:MaxRAMPercentage=85/90` usually. I'll need to think about it more, may be you make sense here. thank you for the follow up – Eugene Feb 17 '22 at 11:04

2 Answers2

2

I was in the same case after upgrading AKS, it appears cgroups v2 awareness has been backported to Java 8u361: https://www.oracle.com/java/technologies/javase/8u361-relnotes.html

I haven't tested it yet but it should solve the problem.

Update 2023-04-11: I have tested various Java 8 JRE images and the only ones that appear to have backported cgroups v2 support are:

In particular, Eclipse Temurin, Azul Zulu, Amazon Corretto and Bellsoft Liberica do not properly recognize cgroups v2.

Olivier Gérardin
  • 1,113
  • 14
  • 27
  • 1
    OpenJDK 8u372 also support cgroup v2. https://bugs.openjdk.org/browse/JDK-8230305 – PSKP Jun 22 '23 at 12:27
0

It's for sure something I do not like, but for images that have a cgroup v2, it seems like I will start it with:

-XX:MaxRAM=$(cat /sys/fs/cgroup/memory.max)

This way, running:

java -XX:MaxRAMPercentage=50.0
     -XX:InitialRAMPercentage=50.0
     // we don't use this one, but just to look at he resident memory
     -XX:+AlwaysPreTouch
     -XX:MaxRAM=$(cat /sys/fs/cgroup/memory.max) 
     Dummy

will show top -o %MEM as 1GB. Not ideal, but works.

Eugene
  • 117,005
  • 15
  • 201
  • 306
  • Hi Eugune, some related information for you. -XX:MaxRAMPercentage will not work with Java8 when running in a container until JDK-8230305 is backported. You can find more information here: https://github.com/corretto/corretto-11/issues/228 – Derek Troy-West May 13 '22 at 09:38