docker mysql in kuberneted ERROR 2005 (HY000): Unknown MySQL server host '' (-3)

Question

I'm trying to deploy a multi-container docker app (https://github.com/shadowHawkeye/eramba). This is the yaml file I'm using to kubectl apply -f

The two images I have (one for DB and one for app) are built docker build -t <> . from the GitHub repo.

The DB_ENV_MYSQL_HOST, I've tried both and <eramba-db.eramba-1>

apiVersion: apps/v1
kind: Deployment
metadata:
  name: eramba
  namespace: eramba-1
  labels:
   app: eramba               
spec:
  replicas: 1
  selector:
    matchLabels:
      app: eramba
  template:
    metadata:
      labels:
        app: eramba
    spec:
      containers:
      - name: eramba
        image: docker.io/deveramba/eramba:latest
        ports:
        - containerPort: 80
        env:
        - name: DB_ENV_MYSQL_DATABASE
          value: "eramba-db"
        - name: DB_ENV_MYSQL_HOST
          value: "eramba-host"
        - name: DB_ENV_MYSQL_USER
          value: "eramba"
        - name: DB_ENV_MYSQL_PASSWORD
          value: "password"
        - name: DB_ENV_MYSQL_ROOT_PASSWORD
          value: "password"
        - name: ERAMBA_HOSTNAME
          value: localhost
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: eramba-db
  namespace: eramba-1
  labels:
   app: eramba-db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: eramba-db
  template:
    metadata:
      labels:
        app: eramba-db
    spec:
      containers:
      - name: eramba-db
        image: docker.io/deveramba/eramba-db:latest
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_DATABASE
          value: "eramba-db"
        - name: MYSQL_USER
          value: "eramba"
        - name: MYSQL_PASSWORD
          value: "password"
        - name: MYSQL_ROOT_PASSWORD
          value: "password"
---
apiVersion: v1
kind: Service
metadata:
  name: db
  namespace: eramba-1
spec:
  selector:
    app: eramba-db
  ports:
    - name: sql
      port: 3306
      targetPort: 3306
---
apiVersion: v1
kind: Service
metadata:
  name: eramba-np
  namespace: eramba-1
spec:
  type: NodePort
  selector:
    app: eramba
  ports:
    - name: http
      port: 80
      targetPort: 80
      nodePort: 30045

The deployment looks like (pods and services output)

root@osboxes:/home/osboxes/manifests# kubectl get pods -n eramba-1
NAME                         READY   STATUS    RESTARTS      AGE
eramba-7f7c88c9d6-zqnzr      1/1     Running   2 (73s ago)   7m47s
eramba-db-6c5fdfb7b8-wtgqd   1/1     Running   0             7m47s

root@osboxes:/home/osboxes/manifests# kubectl get service -o wide -n eramba-1
NAME        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE     SELECTOR
db          ClusterIP   10.98.169.229   <none>        3306/TCP       3h31m   app=eramba-db
eramba-np   NodePort    10.97.149.116   <none>        80:30045/TCP   3h31m   app=eramba

The problem is that kubectl logs <> is complaining unknown host Eramba-host. I've defined DB_ENV_MYSQL_HOST and MYSQL_HOST in both app and db deployments, respectively.

root@osboxes:/home/osboxes/manifests# kubectl logs eramba-7f7c88c9d6-zqnzr -n eramba-1
[i] pre-exec.d - processing /scripts/pre-exec.d/010-apache.sh
tail: can't open '/var/log/apache2/*log': No such file or directory
[i] pre-exec.d - processing /scripts/pre-exec.d/020-eramba-initdb.sh
[i] Waiting for database to setup...
[i] Trying to connect to database: try 1...
ERROR 2005 (HY000): Unknown MySQL server host 'eramba-host' (-3)
[i] Trying to connect to database: try 2...
ERROR 2005 (HY000): Unknown MySQL server host 'eramba-host' (-3)
[i] Trying to connect to database: try 3...
ERROR 2005 (HY000): Unknown MySQL server host 'eramba-host' (-3)
[i] Trying to connect to database: try 4...
ERROR 2005 (HY000): Unknown MySQL server host 'eramba-host' (-3)
[i] Trying to connect to database: try 5...
ERROR 2005 (HY000): Unknown MySQL server host 'eramba-host' (-3)
[i] Trying to connect to database: try 6...
ERROR 2005 (HY000): Unknown MySQL server host 'eramba-host' (-3)
[i] Trying to connect to database: try 7...
ERROR 2005 (HY000): Unknown MySQL server host 'eramba-host' (-3)

Here's the kubectl logs output for the db

root@osboxes:/home/osboxes/manifests# kubectl logs eramba-db-6c5fdfb7b8-wtgqd -n eramba-1
2022-01-07 19:17:00+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.5+maria~focal started.
2022-01-07 19:17:00+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2022-01-07 19:17:00+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.5+maria~focal started.
2022-01-07 19:17:00+00:00 [Note] [Entrypoint]: Initializing database files
2022-01-07 19:17:00 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.


PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
To do so, start the server, then issue the following command:

'/usr/bin/mysql_secure_installation'

which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.

See the MariaDB Knowledgebase at https://mariadb.com/kb or the
MySQL manual for more instructions.

Please report any problems at https://mariadb.org/jira

The latest information about MariaDB is available at https://mariadb.org/.
You can find additional information about the MySQL part at:
https://dev.mysql.com
Consider joining MariaDB's strong and vibrant community:
https://mariadb.org/get-involved/

2022-01-07 19:17:01+00:00 [Note] [Entrypoint]: Database files initialized
2022-01-07 19:17:01+00:00 [Note] [Entrypoint]: Starting temporary server
2022-01-07 19:17:01+00:00 [Note] [Entrypoint]: Waiting for server startup
2022-01-07 19:17:01 0 [Note] mariadbd (server 10.6.5-MariaDB-1:10.6.5+maria~focal) starting as process 96 ...
2022-01-07 19:17:01 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2022-01-07 19:17:01 0 [Note] InnoDB: Number of pools: 1
2022-01-07 19:17:01 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2022-01-07 19:17:01 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2022-01-07 19:17:01 0 [Note] InnoDB: Using Linux native AIO
2022-01-07 19:17:01 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2022-01-07 19:17:01 0 [Note] InnoDB: Completed initialization of buffer pool
2022-01-07 19:17:01 0 [Note] InnoDB: 128 rollback segments are active.
2022-01-07 19:17:01 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-01-07 19:17:01 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2022-01-07 19:17:01 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2022-01-07 19:17:01 0 [Note] InnoDB: 10.6.5 started; log sequence number 41361; transaction id 14
2022-01-07 19:17:01 0 [Note] Plugin 'FEEDBACK' is disabled.
2022-01-07 19:17:01 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2022-01-07 19:17:01 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
2022-01-07 19:17:01 0 [Warning] 'user' entry 'root@eramba-db-6c5fdfb7b8-wtgqd' ignored in --skip-name-resolve mode.
2022-01-07 19:17:01 0 [Warning] 'proxies_priv' entry '@% root@eramba-db-6c5fdfb7b8-wtgqd' ignored in --skip-name-resolve mode.
2022-01-07 19:17:01 0 [Note] InnoDB: Buffer pool(s) load completed at 220107 19:17:01
2022-01-07 19:17:01 0 [Note] mariadbd: ready for connections.
Version: '10.6.5-MariaDB-1:10.6.5+maria~focal'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
2022-01-07 19:17:02+00:00 [Note] [Entrypoint]: Temporary server started.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
2022-01-07 19:17:03 5 [Warning] 'proxies_priv' entry '@% root@eramba-db-6c5fdfb7b8-wtgqd' ignored in --skip-name-resolve mode.
2022-01-07 19:17:03+00:00 [Note] [Entrypoint]: Creating database eramba-db
2022-01-07 19:17:03+00:00 [Note] [Entrypoint]: Creating user eramba
2022-01-07 19:17:03+00:00 [Note] [Entrypoint]: Giving user eramba access to schema eramba-db

2022-01-07 19:17:03+00:00 [Note] [Entrypoint]: Stopping temporary server
2022-01-07 19:17:03 0 [Note] mariadbd (initiated by: root[root] @ localhost []): Normal shutdown
2022-01-07 19:17:03 0 [Note] InnoDB: FTS optimize thread exiting.
2022-01-07 19:17:03 0 [Note] InnoDB: Starting shutdown...
2022-01-07 19:17:03 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
2022-01-07 19:17:03 0 [Note] InnoDB: Buffer pool(s) dump completed at 220107 19:17:03
2022-01-07 19:17:04 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2022-01-07 19:17:04 0 [Note] InnoDB: Shutdown completed; log sequence number 42335; transaction id 15
2022-01-07 19:17:04 0 [Note] mariadbd: Shutdown complete

2022-01-07 19:17:04+00:00 [Note] [Entrypoint]: Temporary server stopped

2022-01-07 19:17:04+00:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up.

2022-01-07 19:17:04 0 [Note] mariadbd (server 10.6.5-MariaDB-1:10.6.5+maria~focal) starting as process 1 ...
2022-01-07 19:17:04 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2022-01-07 19:17:04 0 [Note] InnoDB: Number of pools: 1
2022-01-07 19:17:04 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2022-01-07 19:17:04 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2022-01-07 19:17:05 0 [Note] InnoDB: Using Linux native AIO
2022-01-07 19:17:05 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2022-01-07 19:17:05 0 [Note] InnoDB: Completed initialization of buffer pool
2022-01-07 19:17:05 0 [Note] InnoDB: 128 rollback segments are active.
2022-01-07 19:17:05 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-01-07 19:17:05 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2022-01-07 19:17:05 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2022-01-07 19:17:05 0 [Note] InnoDB: 10.6.5 started; log sequence number 42335; transaction id 14
2022-01-07 19:17:05 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2022-01-07 19:17:05 0 [Note] Plugin 'FEEDBACK' is disabled.
2022-01-07 19:17:05 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
2022-01-07 19:17:05 0 [Note] InnoDB: Buffer pool(s) load completed at 220107 19:17:05
2022-01-07 19:17:05 0 [Note] Server socket created on IP: '0.0.0.0'.
2022-01-07 19:17:05 0 [Note] Server socket created on IP: '::'.
2022-01-07 19:17:05 0 [Warning] 'proxies_priv' entry '@% root@eramba-db-6c5fdfb7b8-wtgqd' ignored in --skip-name-resolve mode.
2022-01-07 19:17:05 0 [Note] mariadbd: ready for connections.
Version: '10.6.5-MariaDB-1:10.6.5+maria~focal'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution

The `ebrama-db` deployment is exposed through the service `db`, hence we should set the `DB_ENV_MYSQL_HOST` to `db`. — Turing85, Jan 07 '22 at 19:29
I am not quite sure what this environment variable does. I operated under the assumption that `ebrama-db` is the database deployment (.e. provides the mysql-pods). I am not entirely sure why the database deployment needs its own hostname. But if the above holds true, then I guess yes. — Turing85, Jan 07 '22 at 19:38
Hey you're right. I removed MYSQL_HOST as it does not need to connect to itself. It provides the mysql-pod. I've changed DB_ENV_MYSQL_HOST to db and also tried eramba-db.eramba-1 (both still says Unknown MySQL server host) — Bryan, Jan 07 '22 at 19:45
Just an FYI you typically want to use a statefulset instead of a deployment for databases. In your current configuration, you will lose data on pod deletion. — jordanm, Jan 07 '22 at 19:48
Do you also have a [Service](https://kubernetes.io/docs/concepts/services-networking/service/) pointing at the database Pod? DNS names resolve to Services, and Services point to Pods, and that's how you normally connect between things in Kubernetes. — David Maze, Jan 08 '22 at 02:06
I do have a service pointing to the db pod yea I think (I've added the kubectle get service output above). I don't have dns setup tho (or any resolution services). Is that what you mean? Thanks, — Bryan, Jan 08 '22 at 02:10

gohm'c · Answer 1 · 2022-01-09T06:23:07.040

4

Here's how you can run Eramba community edition on K8s:

Base on eramba-community-docker. Lots of hardwork by this author, do give the repo a star.
Tested on Linux only.
The MariaDB store data at your host path /tmp/erambadb. You can upgrade it to other storage media as you like.
Address implementation pertain to K8s only. Does not address any eramba specific topic or working.
Run in "default" namespace.
Run eramba web application as Pod. You can upgrade it to Deployment as you like.

First, use your favorite editor to start a eramba-cm.yaml file:

apiVersion: v1
kind: ConfigMap
metadata:
  name: eramba
data:
  c2.8.1.sql: |
    CREATE DATABASE IF NOT EXISTS erambadb;
    USE erambadb;
    ## IMPORTANT: MUST BE INDENT 2 SPACES AFTER c2.8.1.sql ##
    <copy & paste content from here: https://raw.githubusercontent.com/markz0r/eramba-community-docker/master/sql/c2.8.1.sql>

kubectl create -f eramba-cm.yaml

Create the storage for MariaDB:

cat << EOF > eramba-storage.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: eramba-storage
spec:
  storageClassName: eramba-storage
  capacity:
    storage: 5Gi
  accessModes: 
  - ReadWriteOnce
  hostPath:
    path: /tmp/erambadb
    type: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: eramba-storage
spec:
  storageClassName: eramba-storage
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
...
EOF

kubectl create -f eramba-storage.yaml

Install bitnami/mariadb using Helm

helm repo add bitnami https://charts.bitnami.com/bitnami
helm upgrade -i eramba bitnami/mariadb --set auth.rootPassword=eramba,auth.database=erambadb,initdbScriptsConfigMap=eramba,volumePermissions.enabled=true,primary.persistence.existingClaim=eramba-storage

Run eramba web application:

cat << EOF > eramba-web.yaml
apiVersion: v1
kind: Pod
metadata:
  name: eramba-web
  labels:
    app.kubernetes.io/name: eramba-web
spec:
  containers:
  - name: eramba-web
    image: markz0r/eramba-app:c281
    imagePullPolicy: IfNotPresent
    env:
    - name: MYSQL_HOSTNAME
      value: eramba-mariadb
    - name: MYSQL_DATABASE
      value: erambadb
    - name: MYSQL_USER
      value: root
    - name: MYSQL_PASSWORD
      value: eramba
    - name: DATABASE_PREFIX
      value: ""
    ports:
    - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: eramba-web
  labels:
    app.kubernetes.io/name: eramba-web
spec:
  ports:
  - name: http
    nodePort: 30045
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app.kubernetes.io/name: eramba-web
  type: NodePort
...
EOF

Check all that required: kubectl get cm,pvc,pv,svc,pods

You can now browse eramba-web via port-forward or http://<host ip>:30045.

kubectl port-forward service/eramba-web 8888:8080

edited Jan 09 '22 at 06:23

answered Jan 08 '22 at 10:30

gohm'c

13,492
1
9
16

Hey - first, thank you so much for this answer. I'm seeing a problem with the erambada pod where it's saying mariadb 18:32:16.68 ERROR ==> Failed executing /docker-entrypoint-initdb.d/c2.8.1.sql - any ideas why this is happening? – Bryan Jan 08 '22 at 18:37
Omg I think its the configMap. I have to paste the .sql - so sorry lol lemme try – Bryan Jan 08 '22 at 19:11
http://``:30045 -- node IP is your machine IP address. [How to find your host IP address](https://stackoverflow.com/questions/13322485/how-to-get-the-primary-ip-address-of-the-local-machine-on-linux-and-os-x). – gohm'c Jan 09 '22 at 02:24
Thank you - I appreciate it. Btw if I want to create two namespaces under one node (each with one instance of the application or deployment), how would I configure that? Right now, the node IP is the same for both namespaces. – Bryan Jan 09 '22 at 02:27
"namespace" is not specific to any node. You can create as many namespaces as you need. In each of your deployment you specify the desired `namespace` under `metadata` and the resource will be deploy into that namespace. For Helm, you add `--namespace ` to the install command and the mariadb will be deploy into that namespace. For mariadb, be mindful of the storage location, should you continue to use hostPath, you need separate PersistentVolume/PersistentVolumeClaim pair with different location for **every** mariadb deployed. – gohm'c Jan 09 '22 at 02:47
Lastly is the nodePort, don't hardcode it and let K8s pick one for you automatically. You can find the selected port with `kubectl get svc eramba-web --namespace `, look at `PORT` column. When you browse with the specific node port number, the eramba web in that namespace will response to you. – gohm'c Jan 09 '22 at 02:55
so do I just leave nodeport: as empty? – Bryan Jan 09 '22 at 03:17
Omit the field will do. – gohm'c Jan 09 '22 at 03:21
Btw - accessing via nodeIP:PORT returns connection refused. I looked at https://stackoverflow.com/questions/54996210/cant-access-kubernetes-service-exposed-via-nodeport?rq=1 seems like a similar issue. What's your take on it? – Bryan Jan 09 '22 at 04:00
Accessing it via the node IP works tho – Bryan Jan 09 '22 at 04:04
The uff firewall is also disabled – Bryan Jan 09 '22 at 04:38
Do you run your k8s using a VM? – gohm'c Jan 09 '22 at 05:11
Yea its on ubuntu – Bryan Jan 09 '22 at 05:11
Use this VM's IP address. – gohm'c Jan 09 '22 at 05:12
Via the NAT network adapter – Bryan Jan 09 '22 at 05:12
ifconfig turns up two additional IP (10.20.0.1 and 172.17.0.1) - both can reach the web-app but dies when :nodePort is used. I'm using VMWare and the network config where the virtual machine does not have its own IP address on the external network. The virtual machine obtains a private IP address from the VMware virtual DHCP server. – Bryan Jan 09 '22 at 05:18
what do you mean "dies"? See [here](https://docs.vmware.com/en/VMware-Workstation-Pro/16.0/com.vmware.ws.using.doc/GUID-E146C894-664C-479A-9E19-484400614BED.html) for port-forwarding using VMware workstation (assumed). – gohm'c Jan 09 '22 at 05:23
oh sorry as in it just shows "unable to connect. Curl shows connection refused". – Bryan Jan 09 '22 at 05:25
ifconfig on my host shows the network as bridge101 tho – Bryan Jan 09 '22 at 05:25
Base on what you said, your VM is **not** on the same network as your host (eg. your laptop), you cannot curl the VM when it is behind VMware network. See here for [port-forwarding using VMware workstation](https://docs.vmware.com/en/VMware-Workstation-Pro/16.0/com.vmware.ws.using.doc/GUID-E146C894-664C-479A-9E19-484400614BED.html) (assumed). – gohm'c Jan 09 '22 at 05:28
Sorry I mean doing curl on my VM (not host). I'm not trying to reach nodeIP:nodePort via my host Mac even tho the IP can reach the web-app (weird). I'll look into this in the meantime - thanks again :) – Bryan Jan 09 '22 at 05:31
It's all depends on the [network mode](https://docs.vmware.com/en/VMware-Workstation-Pro/16.0/com.vmware.ws.using.doc/GUID-3B504F2F-7A0B-415F-AE01-62363A95D052.html) you used for the VM. In short, in the case for **laptop to VM**, if you use NAT, you need to port-forward. If you use Bridge, the VM is treated like another computer on the same network as your laptop - which in this case you can curl using the VM primary IP like http://:30045. All **assumed** no Firewall in between, otherwise of course you need to open port. – gohm'c Jan 09 '22 at 05:40
So switching to bridged on VMware resulted in Unable to connect to the server: dial tcp 172.16.42.135:6443: i/o timeout (when running kubectl get pods, or any ops). I did see ifconfig docker0 interface was down tho, tried brining it up and fails (not sure if its relevant) – Bryan Jan 09 '22 at 16:00
Oh the docker0 interface is also down for NAT network. – Bryan Jan 09 '22 at 16:02
If you are not sure about the relationship among k8s api-server `--apiserver-advertise-address`, the VM network and your host network, it's better to stick to the original network that you used to setup your k8s cluster. Switch of your VM network does not means K8s will automatically follow thru. If not mistaken you started with VMWare NAT network, as mentioned above; just use the workstation port-forward feature will save you a lot of time. – gohm'c Jan 09 '22 at 16:18
Ahhh thank you! Can you elaborate how to run this port-forward setup? If I want to access the two instances in the two namespace, do I port forward TWO ports (two port-forward commands?). Do I perform the port forwarding on my host or VM? – Bryan Jan 09 '22 at 16:27
One interesting observation is that I can’t port-forward the pod and access via localhost:8888, I can only access via NodeIP – Bryan Jan 09 '22 at 17:20
I've raised this as a separate issue to attach more information - see https://stackoverflow.com/questions/70644537/accessing-via-nodeipnodeport-returns-connection-refused – Bryan Jan 09 '22 at 19:02
Bwt I've figured it out - the selector needs to change to match app.kubernetes.io/name: eramba-web - the application instance is returning "database connection mysql is missing or could not be created. selected driver is not enabled". As this is an application error. I'll look into the database.php config. – Bryan Jan 09 '22 at 23:42
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/240908/discussion-between-bryan-and-gohmc). – Bryan Jan 10 '22 at 02:05

docker mysql in kuberneted ERROR 2005 (HY000): Unknown MySQL server host '' (-3)

1 Answers1

Linked