Makefile docker wait for database to be ready

Question

I'm attempting to create a makefile that will launch my db container, wait for it to complete before launching the rest of my app.

I have 2 compose files.

docker-compose.db.yml
docker-compose.yml

My make file is as follows:

default:
    @echo "Preparing database"
    docker-compose -f docker-compose.db.yml build
    docker-compose -f docker-compose.db.yml pull
    docker-compose -f docker-compose.db.yml up -d

    @echo ""
    @echo "Waiting for database \"ready for connections\""
    @while [ -z "$(shell docker logs $(PROJECT_NAME)_mariadb 2>&1 | grep -o "ready for connections")" ]; \
    do \
        sleep 5; \
    done 
    @echo "Database Ready for connections!"

    @echo ""
    @echo "Launching App Containers"
    docker-compose build
    docker-compose pull
    docker-compose up -d

What happens is that it immediately goes to "Database Ready for connections!" even before the database is ready. If I run the same command in terminal it response with empty for about the first 20 seconds and then finally returns "ready for connections".

Thank you in advance

David Maze · Accepted Answer · 2020-06-23T20:19:39.037

5

The GNU make $(shell ...) function gets run once when the Makefile is processed. So when your rule has

@while [ -z "$(shell docker logs $(PROJECT_NAME)_mariadb 2>&1 | grep -o "ready for connections")" ]

Make first runs the docker logs command on its own, then substitutes the result in the shell command it runs

while [ -z "ready for connections" ]

which is trivially false, and the loop exits immediately.

Instead you probably want to escape the $ in the shell substitution command

@while [ -z "$$(docker-compose logs mariadb ...) "]

It's fairly typical to configure containers to be able to wait for the database startup themselves, and to run the application and database from the same docker-compose.yml file. Docker Compose wait for container X before starting Y describes this setup.

edited Jun 23 '20 at 20:19

answered Jun 23 '20 at 19:39

David Maze

130,717
29
175
215

Sorry, read the direction of the `while` loop wrong. It should either always succeed immediately or never succeed at all because Make runs the command once and inserts the result. – David Maze Jun 23 '20 at 20:20
Ah that makes sense. Didn't realize the `$(shell ...` meant execute at the start. That would make sense why it was getting past the loop because the container was already ready after the first attempt at running the makefile. I am also tearing down the containers on start up with other parts omitted for brevity but that explains a lot. `$$(` for inline it is. Also I've explored using the `docker-compose` and feel like this is more explicit over some of the weird things that you need to do to get it to work in the `docker-compose` although changes easily break this way. – DMCApps Jun 23 '20 at 20:26
It's not quite correct to say that the `shell` function gets run once _when the Makefile is processed_. A shell function in a recipe is run once when the recipe is run, _but_ all make variables and functions in all commands in the recipe are expanded immediately when the rule starts, before the first command in the rule is run. In general it's never a good idea to use a `shell` function in a recipe... why would you need to since a recipe is already running in the shell? It just gives confusion... – MadScientist Jun 23 '20 at 21:50

Makefile docker wait for database to be ready

1 Answers1