1

I'm having some problems running postgres (with data stored on an nfs server) inside a kubernetes cluster.

I think it's related to permissions on the nfs folder. I've changed the user context, but then that user is not recognised in the postgres pod?

I'm slightly confused at this point, and would like a steer on what I'm doing wrong. I assume that I either need to give a postgres user ownership of the folder on my nfs server or give the pod a user called admin?


When I first tried to deploy the pod it failed and gave me the following HINT: The server must be started by the user that owns the data directory.

So I checked the folder on the nfs server and found

user@synology:/volume1/kube_volumes/$ ls -lZ test-db/
total 8
drwxrwxrwx 3 admin users ? 4096 Dec  7 13:19 postgres-data
user@synology:/volume1/kube_volumes/$ id -u admin
1024

I then updated the securityContext to runAsUser: 1024

This allows the pod to start. I then try to exec into the pod, and it seems that user 1024 is unknown

I have no name!@test-db-main-0:/datatoimport$ psql -U postgres -d test-db < test-db-2020_11_09.sql 
psql: error: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I have no name!@test-db-main-0:/datatoimport$ psql -U postgres -d test-db < test-db-2020_11_09.sql 

I've used nfs--external-provisioner with helm to link to my nfs, so the process was:

kubectl create namespace ns-db1
helm install nfs-provisioner stable/nfs-client-provisioner --set storageClass.name=nfs1 --set nfs.server=192.168.1.1 --set nfs.path=/volume1/kube_volumes/test-db --namespace=ns-db1
kubectl create -f pvc.yaml
kubectl create -f db-deployment.yaml

db-deployment.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: test-db-main
  name: test-db-main
  namespace: ns-db1
spec:
  replicas: 1
  serviceName: test-db-main-svc
  selector:
    matchLabels:
      app: test-db-main
  template:
    metadata:
      labels:
        app: test-db-main
    spec:
      securityContext:
        runAsUser: 1024
        supplementalGroups: [999,1000]
        fsGroup: 999
      containers:
      - name: test-db-main
        env:
        - name: PGDATA
          value: /var/lib/postgresql/data/test-db
        - name: POSTGRES_DB
          value: test-db
        - name: POSTGRES_PASSWORD
          value: ****
        - name: POSTGRES_USER
          value: postgres
        image: postgres:13.1
        imagePullPolicy: ""
        resources: {}
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: pvc-test-db
          subPath: test-db-main/postgres-data
        - mountPath: /datatoimport
          name: pvc-test-db
          subPath: test-db-main/datatoimport
      restartPolicy: Always
      serviceAccountName: ""
      volumes:
      - name: pvc-test-db
        persistentVolumeClaim:
          claimName: pvc-test-db
status:
  replicas: 1

pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-test-db
  namespace: ns-db1
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: nfs1
  resources:
    requests:
      storage: 10Gi

EDIT:

full log from pod as requested

2020-12-07T13:21:22.998744283Z The files belonging to this database system will be owned by user "postgres".
2020-12-07T13:21:22.998852354Z This user must also own the server process.
2020-12-07T13:21:22.998875562Z 
2020-12-07T13:21:22.998912661Z The database cluster will be initialized with locale "en_US.utf8".
2020-12-07T13:21:22.998919809Z The default database encoding has accordingly been set to "UTF8".
2020-12-07T13:21:22.998924625Z The default text search configuration will be set to "english".
2020-12-07T13:21:22.998929325Z 
2020-12-07T13:21:22.998933654Z Data page checksums are disabled.
2020-12-07T13:21:22.998938214Z 
2020-12-07T13:21:23.105015515Z fixing permissions on existing directory /var/lib/postgresql/data/test-db ... ok
2020-12-07T13:21:23.533981413Z creating subdirectories ... ok
2020-12-07T13:21:23.571899102Z selecting dynamic shared memory implementation ... posix
2020-12-07T13:21:23.803296593Z selecting default max_connections ... 20
2020-12-07T13:21:25.230873501Z selecting default shared_buffers ... 400kB
2020-12-07T13:21:25.267232138Z selecting default time zone ... Etc/UTC
2020-12-07T13:21:25.503743387Z creating configuration files ... ok
2020-12-07T13:21:25.592990375Z 2020-12-07 13:21:25.592 UTC [68] FATAL:  data directory "/var/lib/postgresql/data/test-db" has wrong ownership
2020-12-07T13:21:25.593033230Z 2020-12-07 13:21:25.592 UTC [68] HINT:  The server must be started by the user that owns the data directory.
2020-12-07T13:21:25.602163505Z child process exited with exit code 1
2020-12-07T13:21:25.602216101Z initdb: removing contents of data directory "/var/lib/postgresql/data/test-db"
2020-12-07T13:21:27.492558318Z running bootstrap script ... 

Changing runAsUser to 999 gives:

PostgreSQL Database directory appears to contain a database; Skipping initialization

2020-12-09 10:50:35.496 UTC [1] FATAL:  data directory "/var/lib/postgresql/data/test-db" has wrong ownership
2020-12-09 10:50:35.496 UTC [1] HINT:  The server must be started by the user that owns the data directory.
yahop19531
  • 193
  • 1
  • 11
  • Can you provide full log from the pod ? Can you change securityContext runAsUser to 999 from 1024? Did you take a look: https://stackoverflow.com/questions/57196736/the-server-must-be-started-by-the-user-that-owns-the-data-directory ? – Malgorzata Dec 09 '20 at 10:32
  • I tried to follow that. I'm not sure how to "ensure the nfs mount is owned by 999:999" though? Have added the log from the pod and the result of runAsUser 999 – yahop19531 Dec 09 '20 at 10:48
  • Can you check after going to shell of your container - execute command: whoami && id -u && id -g ? Also did you check https://stackoverflow.com/questions/42653690/psql-could-not-connect-to-server-no-such-file-or-directory-5432-error https://stackoverflow.com/questions/28925848/error-with-postgresql-datababse-is-the-server-running-locally-and-accepting-co ? Also try to change StatefulSet to Deployment. – Malgorzata Dec 30 '20 at 10:41

0 Answers0