3

I have an openjdk:8 image running on the Kubernetes cluster. I added memory HPA (Horizontal Pod Autoscaling) which scales up fine but since JVM doesn't release the memory back from the heap to the OS, pods do not scale down. Following is the hpa.yaml

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: image-server
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: image-server
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 60

One way to solve this is to use the right GC and make it release the memory, but since JVM has been designed to not release from the heap frequently for performance reasons, doing this isn't a good idea. Is there a way to handle this from Kubernetes? Like instead of checking OS's memory usage, can we not just check the memory usage from heap and scale on that?

danielorn
  • 5,254
  • 1
  • 18
  • 31
Manoj Suthar
  • 1,415
  • 3
  • 19
  • 41
  • you are correct that JVM does not usually gives memory back, but this is not entirely correct either. look [here](https://stackoverflow.com/questions/61506136/kubernetes-pod-memory-java-gc-logs/61512521#61512521) and [here, if you are using G1](https://stackoverflow.com/questions/59362760/does-g1gc-release-back-memory-to-the-os-even-if-xms-xmx/59377080#59377080) – Eugene Aug 29 '21 at 02:54
  • @Eugene I am using openjdk 8 which uses ParallelGC as default. JVM holds on to the memory to avoid the alloc dealloc cycle for performance reasons, by using a different GC or configuring it to release memory more frequently may increase that cycle. And since we only have a java process running on this pod, OS doesn't need it for anything else and it is fine even if Java holds on to it. I was looking for a way to do the HPA based on the heap usage instead of the memory usage reported by the OS. – Manoj Suthar Aug 29 '21 at 18:12
  • you can't. But you can use a diff GC to release memory faster ( or at all ) and thus almost achieve what you want. And the claims that it does not do that because of performance impacts are a myth of the past. Every concurrent GC, like Shenandoah and ZGC ( even G1 as the links show you ), release it. – Eugene Aug 29 '21 at 18:25

1 Answers1

8

Scaling Java applications in Kubernetes is a bit tricky. The HPA looks at system memory only and as pointed out, the JVM generally do not release commited heap space (at least not immediately).

There are two main approaches one could take to solve this

1. Tune JVM Parameters so that the commited heap follows the used heap more closely

Depending on which JVM and GC is in use the tuning options may be slightly different, but the most important ones would be

  • MaxHeapFreeRatio - How much of the commited heap that is allowed to be unused
  • GCTimeRatio - How often GC is allowed to run (impacts performance)
  • AdaptiveSizePolicyWeight - How to weigh older vs newer GC runs when calculating new heap

Giving exact values for these are not easy, it is a compromise between releasing memory fast and application performance. The best settings will be dependant on the load characteristics of the application.

Patrick Dillon has written an article published by RedHat called Scaling Java containers that deep dives into this subject.

2. Custom scaling logic

Instead of using the HPA you could create your own scaling logic and deploy it into Kubernetes as a job running periodically to do:

  1. Check the heap usage in all pods (for example by running jstat inside the pod)
  2. Scale out new pods if the max threshold is reached
  3. Scale in pods if the min threshold is reached

This approach has the benefit of looking at the actual heap usage, but requires a custom component.

An example of this can be found in the article Autoscaling based on CPU/Memory in Kubernetes — Part II by powercloudup

danielorn
  • 5,254
  • 1
  • 18
  • 31
  • or you can [choose a proper GC to begin with](https://stackoverflow.com/questions/61506136/kubernetes-pod-memory-java-gc-logs/61512521#61512521), like we did. – Eugene Aug 29 '21 at 02:52