How can I set `max_map_count` and `ulimit` for Elasticsearch node when running in Kubernete in EKS?

Question

I am deploying Elasticsearch 7.10.1 to AWS EKS Fargate but I got below error when running them:

ERROR: [2] bootstrap checks failed
[1]: max number of threads [1024] for user [elasticsearch] is too low, increase to at least [4096]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

I found solutions for them is max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] and Elasticsearch: Max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144].

But both requires a change on the host machine. I am using EKS Fargate which means I don't have access to the Kubernete cluster host machine. What else should I do to solve this issue?

Answer 1

Your best bet is to set these via privileged init containers within your Elasticsearch pod/deployment/statefulset, for example:

apiVersion: v1
kind: Pod
metadata:
  name: elasticsearch-node
spec:
  initContainers:
    - name: increase-vm-max-map
      image: busybox
      command: ["sysctl", "-w", "vm.max_map_count=262144"]
      securityContext:
        privileged: true
    - name: increase-fd-ulimit
      image: busybox
      command: ["sh", "-c", "ulimit -n 65536"]
      securityContext:
        privileged: true
  containers:
    - name: elasticsearch-node
      ...

You could also do this through Daemonsets, although Daemonsets aren't very well suited to one-time tasks (but it's possible to hack around this). But the init container approach will guarantee that your expected settings are in effect precisely before an Elasticsearch container is launched.