Starting lots of Docker containers occasionally causes address pool exhaustion

Question

I want to start lots of Docker containers for testing purposes. Each test has a private network and a few containers in that network. Communication is only needed within each network. Thus I should be able to run many tests at once.

However, I occasionally get this error:

{"message":"could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network"}

This is surprising since all of the networks are "internal".

What causes this and how do I prevent this from happening?

I am using the Docker CLI.

---

This question mentions a limit on the number of networks. Is there a way to circumvent this limit?

---

docker --version
Docker version 20.10.7, build 20.10.7-0ubuntu5~20.04.2

Answer 1

This could be one of the few cases where it's useful to manually specify IP-related information in your Docker networking setup.

The question you link to further links through to a GitHub issue which eventually names the specific networks Docker uses: the 16 networks 172.17.0.0/16 through 172.32.0.0/16, plus the 16 networks 192.168.0.0/20 through 192.168.240.0/20. (Also see this comment on a related issue.) You say you only are launching a couple of containers on each network, so a /16 subnet (65,534 addresses) or even a /20 (4,094) is excessive.

With the Docker CLI, you can docker network create a network with a specific --subnet, in standard CIDR format. It's your responsibility to make sure the networks don't overlap each other or any other network on your host (or in the Docker VM, if appropriate). Note that Docker is likely to have allocated 172.17.0.0/16 on its own for the "default bridge network" so that may not be available to you.

If you're not otherwise using 172.18.0.0/16, though, you could partition it into 256 /24 networks:

docker network create --subnet=172.18.0.0/24 net0
docker network create --subnet=172.18.1.0/24 net1
...
docker network create --subnet=172.18.255.0/24 net255

Docker will create the gateway and broadcast addresses in the usual way, and assign per-container IP addresses on its own; you do not need the docker run --ip option.