How can I identify the issue preventing a connection between my web application Docker container and the associated MySQL Docker container?

Question

I am trying to deploy a small personal website (and backing MySQL) to Docker using Docker Compose. While I have managed to get the web application running on Docker, I cannot get it to connect to the database. This project is the first time I have been using Docker, so I believe I cannot spot the issue due to my inexperience. I wonder if anyone more experienced could see what I am doing wrong.

I have been going at it for days now. However, none of the solutions to similar questions has worked for me. So far, I understand that the 'Communications link failure' error I keep getting is very generic. Thus, it is difficult to pinpoint the exact problem.

I have provided the contents of my Dockerfile, docker-compose.yml, the database configurations in my application.properties, and part of the output log. I have also provided a Pastebin URL for the entire log.

Many Thanks.

Complete Log Output: https://pastebin.com/aYjWnck8

Dockerfile

FROM openjdk:17
COPY target/personal-website-0.0.1-SNAPSHOT.jar personal-website-0.0.1-SNAPSHOT.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "personal-website-0.0.1-SNAPSHOT.jar"]

docker-compose.yml

version: "3.8"
services:
  mysqldb:
    container_name: personal-website-database
    image: mysql:8.0.29
    restart: always
    ports:
      - 3307:3306
    environment:
      MYSQL_DATABASE: skills
      MYSQL_USER: chizzy
      MYSQL_PASSWORD: AbcXyz
      MYSQL_ROOT_PASSWORD: AbcXyz
  server:
    build: .
    container_name: personal-website
    restart: always
    ports:
      - 8081:8080
    environment:
      SPRING_DATASOURCE_URL: jdbc:mysql://mysqldb:3306/skills?useSSL=false&allowPublicKeyRetrieval=true
    depends_on:
      - mysqldb

application.properties (Database Configuration Excerpt)

server.port=8080
spring.datasource.url=jdbc:mysql://localhost:3306/skills
spring.datasource.username=root
spring.datasource.password=AbcXyz
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL8Dialect
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true

Log (Partial)

personal-website           | Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
personal-website           |
personal-website           | The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
personal-website           |    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:na]
personal-website           |    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[na:na]
personal-website           |    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:na]
personal-website           |    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[na:na]
personal-website           |    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[na:na]
personal-website           |    at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:89) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    at com.mysql.cj.NativeSession.connect(NativeSession.java:120) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:948) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    ... 57 common frames omitted
personal-website           | Caused by: java.net.ConnectException: Connection refused
personal-website           |    at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
personal-website           |    at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672) ~[na:na]
personal-website           |    at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542) ~[na:na]
personal-website           |    at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[na:na]
personal-website           |    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) ~[na:na]
personal-website           |    at java.base/java.net.Socket.connect(Socket.java:633) ~[na:na]
personal-website           |    at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:153) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:63) ~[mysql-connector-java-8.0.29.jar!/:8.0.29]
personal-website           |    ... 60 common frames omitted

In the extended logs, are there database startup messages after this application exception? If you run `docker-compose up -d` a second time after waiting a minute or so, is it successful? — David Maze, Oct 01 '22 at 16:21
Hey @DavidMaze, yes, there are. I can see that it creates the 'skills' database alongside the user 'chizzymeka' and grants the user access to the database right after the exception. I have also tried the 'docker-compose up' a second time, that is, while the containers are already running. I noticed that the exception does not appear on the subsequent execution of the command. Do you have any insights as to what could be happening, please? — Chizzy Meka, Oct 01 '22 at 17:23
The database can take 30-60 seconds to start up, especially the first time you start it, but often times the application will start faster. When this happens you get exactly the symptoms you describe: a "connection refused" error starting the stack, but success if you run `docker-compose up -d` again. The linked question describes two ways to get around this, either adding a health check to the Compose dependency or adding a script to the container startup to wait for the database. — David Maze, Oct 01 '22 at 18:03
Thank you very much, @DavidMaze. Your initial question made me suspicious of that fact. Then Mihai's comments below strongly made me believe that that was the problem. No worries, I am working on it and will let you know the outcome. — Chizzy Meka, Oct 01 '22 at 18:35
Thank you very much again for your assistance, @DavidMaze. I have now resolved the issue using a condition that monitors the outcome of a healthcheck whereby the test is: mysqladmin ping -h 127.0.0.1 -u $$MYSQL_USER --password=$$MYSQL_PASSWORD — Chizzy Meka, Oct 01 '22 at 20:08

score 1 · Answer 1 · answered Oct 01 '22 at 17:14

When you start services with docker-compose, they start at the same time. Even if you use depends_on that only insures that the service it depends on receive the start signal, not that it is actually ready to receive connections.

A java application starts quicker than a database. Unfortunately java also tries to connect to the database during startup and doesn't retry if that fails.

You should have a retry mechanism in your application or a waiting loop in your docker-compose to fix this. The waiting loop would delay the starting of the java application until the database is actually ready to receive requests. The retry mechanism in the Java application would try to connect to the database everytime it fails because a connection is not readily available.

I suggest the wait loop and you can find a very good example here: https://docs.docker.com/compose/startup-order/

I could paste that code here but it would be a good practice for you to actually read the article there and implement it. You only need to change the command of your java application and add the wait-for-it functionality (you need the script as well in the docker image ;) )

Thank you, Mihai. I will explore it and let you know the outcome. — Chizzy Meka, Oct 01 '22 at 17:25
Thank you very much again for your assistance, @Mihai. I have now resolved the issue using a condition that monitors the outcome of a healthcheck whereby the test is: mysqladmin ping -h 127.0.0.1 -u $$MYSQL_USER --password=$$MYSQL_PASSWORD — Chizzy Meka, Oct 01 '22 at 20:07

How can I identify the issue preventing a connection between my web application Docker container and the associated MySQL Docker container?

1 Answers1