52

I have a project with a dockerfile and a docker-compose.yml for deploying it. But if I run docker-compose up, the command prints Building app and hangs forever. I do not see any steps being performed. It isn't downloading images either since there's no network traffic, and the images used are available in the cache.

How can I go about debugging and fixing this issue on my machine? Is there a verbose option for docker(-compose) that I could use?

Bonifacio2
  • 3,405
  • 6
  • 34
  • 54
fctorial
  • 765
  • 1
  • 6
  • 11

12 Answers12

147

tl;dr

If Docker Compose is hanging unexpectedly:

  • Make sure you're using Docker Compose v2.
  • Check your system entropy and apt install haveged if it's low.
  • Expect low entropy on cloud / virtualized hardware.

Update (February 2023)

Docker Compose v2 has been rewritten in Go.

I'm still seeing reports of people experiencing hangs within the last 12 months, but I can't verify that those reports are from people running Compose v1 or v2.

My understanding is that Compose v1 hangs for all commands due to a core dependency that needs entropy to initialize, preventing even --help and --version from running.

Contrast that with v2, which only appears to use randomness when generating container names.

Docker itself uses randomness more, so I would imagine it is more likely to hang at this point than Compose.

So - haveged is still your friend if you're running on any virtualized hardware or are experiencing entropy-related hangs, as described below - but if you're experiencing this in 2023, make sure you're using Compose v2.

Note that Docker Compose v1 is now unsupported as of July 2023. It also hasn't received security updates since May 2021, so start migrating now.

Original (June 2021):

As this is the top hit for "docker-compose hangs", I'll post the solution for my problem that would have saved me 30 minutes of searching. If even docker-compose --version hangs, this might be your fix.

Docker-compose seems to hang indefinitely when the system has low entropy. You can check this by running:

cat /proc/sys/kernel/random/entropy_avail

If the output is less than 1000, that is almost certainly why it's hanging. I was experiencing this on a VPS, where the available entropy was less than 10.

The fix for me was to install haveged, which immediately increased my available entropy to over 2500, and caused docker compose to work normally.

Just run apt install haveged or your distro's equivalent and you're good to go.

Phillip Elm
  • 2,094
  • 2
  • 16
  • 27
  • 8
    Thanks, this worked for me, but I've never had this problem before on the (virtual) machine where it happened. Why would this happen? – Will Vousden Jul 03 '21 at 14:45
  • 1
    @WillVousden my guess would be that a process on the host was draining the entropy pool faster than it could be replenished. IIRC, entropy can take a while to accumulate, especially on virtualized hardware. – Phillip Elm Jul 05 '21 at 06:24
  • 34
    Legend. I would not have guessed this in a million years. Thank you. – hraban Jul 06 '21 at 02:14
  • 2
    `[sudo] apt-get install haveged` – Tulleb Sep 09 '21 at 16:18
  • 6
    Jumped from 68 to 2238 after the install and docker-compose has been significantly more responsive since. This answer wins my vote for best of 2021 as I would have NEVER guessed this approach. – Chris Porter Dec 21 '21 at 21:08
  • 2
    you are legend. we don't upvote enough the question and the answer these days. – Calvintwr Jan 11 '22 at 11:27
  • 2
    Wow, no comment. I'd be interested in learning why this helps, but it definitely does. @everyone-at-stack-overflow don't forget to upvote!!! – milosmns Feb 06 '22 at 11:07
  • 3
    @milosmns - thanks! yeah this one came out of left field. it's been a while - but iirc, compose makes a call to libsodium that blocks until there's enough entropy - even when it doesn't really seem to need it (such as the `--version` command). It seems that in some cases, virtual private servers don't generate enough entropy to ever satisfy this condition, so the process hangs indefinitely. – Phillip Elm Feb 08 '22 at 09:50
  • 1
    I am at a loss for words. How bizarre. Thank you. – Tobias Fünke Feb 15 '22 at 07:33
  • 1
    thanks for this! – Martin Melka Feb 01 '23 at 12:02
  • Probably a noob question but this solution isn't for Windows, is it? I'm experiencing the same hang but on a Windows machine. – senecaTheMeek Feb 08 '23 at 13:36
  • @senecaTheMeek - from my understanding, `haveged` is Linux only. Are you using WSL? If so, you can install it on Linux hosts. Also - is your Docker Compose up to date? The original Python version of Docker Compose had a dependency that waited for entropy before Compose could even start - whereas the new Go version of the binary handles randomness completely different. So if you're hanging on commands like `docker-compose --version`, you're probably running an old Python build. – Phillip Elm Feb 14 '23 at 21:57
  • Hi everyone, I'm still having a similar issue. When running ```cat /proc/sys/kernel/random/entropy_avail```, is 256 considered high enough?, it seems to be the upper limit. I've created a new post for my issue here.. https://stackoverflow.com/questions/76196599/docker-compose-up-hangs-forever-usually-even-with-v2-despite-high-entropy – Latchy May 08 '23 at 09:06
  • 1
    @Latchy Hmm - yeah on my local machine I'm showing 256 as well. Doing a quick search, it seems some Linux kernel updates might be causing the entropy number change: https://askubuntu.com/a/1427476/617358 - how it relates to Compose v2 hanging, I'm not sure. This might not even be related to randomness. My first thought would be to double check that Docker is running and healthy, and then run the command with `--verbose` to see where it's getting stuck. If that fails, `strace` could help too. – Phillip Elm May 09 '23 at 04:17
  • @PhillipElm Thanks for the reply, I'll look into this and let you know how I get on. I agree though, in that my issue might not even be related to entropy. – Latchy May 15 '23 at 17:38
  • tried installing haveged but I'm still stucked at "naming to..." as @Latchy said: looks like after kernel 5.4 linux has a hardcoded 256 value. Also: my machine is not virtualized. – ZeoS Jun 07 '23 at 13:20
32

Try to run it with this option and see if helps:

docker-compose --verbose up

*Note: If you haven't build your container, run the same command with the option --build

Felipe Malara
  • 1,796
  • 1
  • 7
  • 13
  • 1
    I had to remove the dash for it to work: `docker compose --verbose up` – Alvaro Rodriguez Scelza Jan 13 '22 at 19:55
  • 1
    @AlvaroRodriguezScelza `docker-compose` and `docker compose` are two different tools for doing the same thing. – Lenormju Jul 06 '22 at 07:01
  • This solved my problem by showing that my docker buildx context was set to build my images using two servers. One of those servers was a remote endpoint, configured to be accessible via SSH. However, that server was down. By running `docker compose --verbose build`, I could immediately see that docker compose was attempting to access an inaccessible endpoint. – ewilan Dec 14 '22 at 00:17
13

I ran into this problem last night, the problem for me was a large sub directory of images, 66GB. Adding the sub directory to .dockerignore file resolved the issue

gordonbanderson
  • 221
  • 2
  • 5
  • 1
    This fixed my problems with version 1.27 of docker-compose. My fix was to create a .dockerignore file to the same directory with my docker-compose.yml file and include the name of the "big" subdir in it. – Robert McLean Aug 09 '21 at 16:36
  • 1
    adding a .dockerignore file and ignoring a large directly (10+gb) also fixed it for me. thanks – Scratcha Oct 18 '21 at 13:17
  • This answer should be higher up. The accepted answer did not help at all - this was exactly the problem for me. – BruceHill Oct 10 '22 at 08:00
3

If you are in a country like Iran probably the problem is sanction you can fix it using VPN or custom anti sanction DNS like

178.22.122.100
185.51.200.2 
OR 
185.55.225.25
185.55.226.26

or you can use unofficial docker registries built for crossing sanction limits like https://docker-registry.ir/

Mahdi mehrabi
  • 1,486
  • 2
  • 18
  • 21
2

I have recently faced that issue this is how I fixed it. There can be multiple reasons for it. But in my case it was because of RAM and memory issue this is how I fixed it

  • Increased RAM from 2 to 4GB
  • Increase Swap memory from 1 to 2GB

If you don't have any such problem, better to verbose it. That will show you the interactive logs on the console

Or you can see the logs of the container

docker logs --tail=200 bootcamp
Umar Asghar
  • 3,808
  • 1
  • 36
  • 32
2

Sometimes, the problem is only about your environment...

Check if there are any containers running on a port that you might be using in your application.

Just use docker ps (as we are interested only in running containers) Check the ports in use If any of them was used by the application try using docker stop [CONTAINER_ID]

Musaab Ghazal
  • 41
  • 1
1

My issue was docker was created multiple networks with the same name:

So run: docker network ls

you'll get the networks list:

d77315668c70   my_network   bridge    local
42db8739a34a   my_network   bridge    local
307198de8065   my_network   bridge    local
217b7c68c345   none         null      local

So try to delete the unused and duplicated networks by their ID:

docker network rm d77315668c70

Mwthreex
  • 913
  • 1
  • 11
  • 24
  • I had various stale networks with different names from playing with K3D, deleting all the non-default networks fixed compose for me based on this answer, thanks! – Geoff Williams Jul 04 '23 at 00:19
0

If you have image builds in docker-compose (not only pulling images) it might be a case when you have too big docker context.

Just add .dockerignore and review all unnecessary folders and exclude them.

Athlan
  • 6,389
  • 4
  • 38
  • 56
0

I ran into the same issue (but on "docker compose" (V2)).

The problem was because I was running the same services (with the same names) on a different project (folder);

So I stopped their containers from running, and the problem was solved.

Behnia FB
  • 53
  • 4
  • 1
    It's funny because again I ran into this problem & forgot the solution until I face my answer here – Behnia FB Jul 03 '23 at 07:12
  • 1
    Haha. Indeed, I faced the issue too last week and again today I had the problem because I again used the same directory structure for my project without taking care of overriding the default Docker Compose project name (see my answer above/below depending on the sorting algorithm =) ) – wget Jul 03 '23 at 16:14
0

On my side, I encountered this issue because several of my Docker Compose recipes where all residing in a directory structure like this one:

/srv/cloud.example.org
/srv/cloud.example.org/docker-compose/docker-compose.yml
/srv/foo.example.org
/srv/foo.example.org/docker-compose/docker-compose.yml

Meaning when I was restarting the Docker Compose recipe for cloud.example.org for example, Docker Compose thought the request concerned also the other Docker Compose recipe.

This is due to the fact, Docker Compose considers the project name as, by default, the name of the parent directory where reside the .yml file, here the directory docker-compose.

To avoid this issue, we can override the Docker Compose project name via the COMPOSE_PROJECT_NAME env variable. More recently (src., src.), directly from the recipe under the top level name: key.

Forcing a project name in each of my recipes fixed my issue and Docker Compose doesn't infinitely hang.

wget
  • 195
  • 2
  • 15
0

In my case it was just a wrong path set as context in the build section of one service.

I would have hoped for an error message like "path/file not found". Clearly my own fault, but not the greatest developer experience from docker-compose.

Clemens
  • 653
  • 6
  • 8
0

Maybe it is too basic, but I ran into this problem with my MacBook Pro and the reason why docker-compose was hanging was because Docker desktop was paused. I don't know why is paused, but restarting it fixed the issue.

Julien Séveno-Piltant
  • 360
  • 1
  • 5
  • 17