Change password in mongodb deployed on kubernetes

Question

I am unable to change the password of an existing user from MongoDB deployed on k8s, unless I am deleting the database and then recreating it again with the new password.

How can I change the password using the yaml for the mongo stateful object without deleting the db?

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongo-db-statefulset
  namespace: development
spec:
  serviceName: mongo-svc
  replicas: 1
  selector:
    matchLabels:
      component: mongo
  template:
    metadata:
      labels:
        component: mongo
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongo
          image: mongo:4.0.4
          volumeMounts:
            - mountPath: /data/db
              name: volume 
          ports:
            - containerPort: 27017
          env:
            - name: MONGO_INITDB_ROOT_USERNAME
              value: admin
            - name: MONGO_INITDB_ROOT_PASSWORD
             # from secrets
              value: password
            - name: MONGO_INITDB_DATABASE
              value: admin
      volumes:
        - name: volume
          persistentVolumeClaim:
            claimName: database-persistent-volume-claim

If you don;t want to delete db, can;t you just change the password using these commands? https://docs.bitnami.com/bch/apps/lets-chat/administration/change-reset-password/ — mchawre, Jan 18 '21 at 17:59
Hi @Mike Me, What you mean by `# from secrets`, MONGO_INITDB_ROOT_PASSWORD is a value from a secret, and you changed it in a secret, or it is specified in the statefulset yaml? — Jakub, Jan 19 '21 at 10:46

Jakub · Accepted Answer · 2021-01-19T12:30:33.670

If I understand your issue correctly:

You have secret with your password as environment variable, and pod has access to the secret data through a Volume
You changed the secret password, but it's not getting picked up by a pod without a restart.

According to documentation:

Environment variables are not updated after a secret update, so if If a container already consumes a Secret in an environment variable, a Secret update will not be seen by the container unless it is restarted. There are third party solutions for triggering restarts when secrets change.

This is a known issue. You can read more about it in this github issue.

So after you change the secret password you have to restart your pod to update this value, you don't have to delete it.

As mentioned in documentation there are third party tools for triggering restart when secrets change, one of them is Reloader.

Reloader can watch changes in ConfigMap and Secret and do rolling upgrades on Pods with their associated DeploymentConfigs, Deployments, Daemonsets and Statefulsets.

The quick way to restart deployment would be to use kubectl rollout restart, which performs a step by step shutdown and restarts each container in your deployment or statefulset.

If you change the password in your secret and use kubectl rollout restart the new password should work.

How do you figure out what to use for RESOURCE in "kubectl rollout restart RESOURCE"? It'd be nice if you provided a full example based on the question details. — Kane, Mar 26 '22 at 17:02

Simu Liviu · Answer 2 · 2023-01-11T14:40:55.877

Example of command for @kane, I don't have reputation to add as a comment: kubectl -n namespace rollout restart deployment/name_of_deployment

Edit: I changed the password in the secret, yet after kubectl rollout restart mongo somehow uses the old password, not sure how. Edit2: If I connect to mongo pod, variable MONGO_INITDB_ROOT_PASSWORD shows the new password, yet connecting to mongo db works only with the old one Edit3: What I ended up doing was: 1. Keep the secret with new value, not sure if it even matters at this point. 2. Login to pod and run this 2 commands: db = db.getSiblingDB('admin') followed by db.changeUserPassword("root", "NEWPASSWORD") 3. Just to test this is persistent, I deleted the deployment and created it again and the new password is working, old one not anymore

Change password in mongodb deployed on kubernetes

2 Answers2