Test Container test cases are failing due to "Could not find a valid Docker environment"

Question

I am very new to using test containers. My test is failing with below exception.

Running com.mastercard.example.testcontainers.testcontainersexampple.DemoControllerTest
2020-04-08 14:27:08.441  INFO   --- [           main] o.s.t.c.support.AbstractContextLoader    
: Could not detect default resource locations for test class 
resource found for suffixes {-context.xml, Context.groovy}.
2020-04-08 14:27:08.449  INFO   --- [           main] t.c.s.AnnotationConfigContextLoaderUtils : Could not detect default configuration classes for test class [com.mastercard.example.testcontainers.testcontainersexampple.DemoControllerTest]: DemoControllerTest does not declare any static, non-private, non-final, nested classes annotated with @Configuration.
2020-04-08 14:27:08.611  INFO   --- [           main] .b.t.c.SpringBootTestContextBootstrapper : Found @SpringBootConfiguration com.mastercard.example.testcontainers.testcontainersexampple.TestContainersExampleApplication for test class com.mastercard.example.testcontainers.testcontainersexampple.DemoControllerTest
2020-04-08 14:27:08.701  INFO   --- [           main] .b.t.c.SpringBootTestContextBootstrapper : Loaded default TestExecutionListener class names from location [META-INF/spring.factories]: [org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener, org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener, org.springframework.boot.test.autoconfigure.restdocs.RestDocsTestExecutionListener, org.springframework.boot.test.autoconfigure.web.client.MockRestServiceServerResetTestExecutionListener, org.springframework.boot.test.autoconfigure.web.servlet.MockMvcPrintOnlyOnFailureTestExecutionListener, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverTestExecutionListener, org.springframework.test.context.web.ServletTestExecutionListener, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener, org.springframework.test.context.support.DependencyInjectionTestExecutionListener, org.springframework.test.context.support.DirtiesContextTestExecutionListener, org.springframework.test.context.transaction.TransactionalTestExecutionListener, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener, org.springframework.test.context.event.EventPublishingTestExecutionListener]
2020-04-08 14:27:08.725  INFO   --- [           main] .b.t.c.SpringBootTestContextBootstrapper : Using TestExecutionListeners: [org.springframework.test.context.web.ServletTestExecutionListener@117159c0, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener@3e27ba32, org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener@7ef82753, org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener@3b0fe47a, org.springframework.test.context.support.DirtiesContextTestExecutionListener@202b0582, org.springframework.test.context.transaction.TransactionalTestExecutionListener@235ecd9f, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener@1ca3b418, org.springframework.test.context.event.EventPublishingTestExecutionListener@58cbafc2, org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener@2034b64c, org.springframework.boot.test.autoconfigure.restdocs.RestDocsTestExecutionListener@75d3a5e0, org.springframework.boot.test.autoconfigure.web.client.MockRestServiceServerResetTestExecutionListener@74d1dc36, org.springframework.boot.test.autoconfigure.web.servlet.MockMvcPrintOnlyOnFailureTestExecutionListener@7161d8d1, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverTestExecutionListener@74e28667]
2020-04-08 14:27:08.781 ERROR   --- [           main] o.t.d.DockerClientProviderStrategy       : Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
2020-04-08 14:27:08.782 ERROR   --- [           main] o.t.d.DockerClientProviderStrategy       :     UnixSocketClientProviderStrategy: failed with exception InvalidConfigurationException (ping failed). Root cause NoSuchFileException (/var/run/docker.sock)
2020-04-08 14:27:08.782 ERROR   --- [           main] o.t.d.DockerClientProviderStrategy       : As no valid configuration was found, execution cannot continue

My question is do I need docker installed locally in order to use testcontainers? If yes, how does this works from pipeline like Jenkins?

Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$3(DockerClientProviderStrategy.java:158)
at java.util.Optional.orElseThrow(Optional.java:290)
at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:150)
at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:111)
at org.testcontainers.containers.GenericContainer.<init>(GenericContainer.java:175)
at org.testcontainers.containers.JdbcDatabaseContainer.<init>(JdbcDatabaseContainer.java:36)
at org.testcontainers.containers.PostgreSQLContainer.<init>(PostgreSQLContainer.java:32)
at com.mastercard.example.testcontainers.testcontainersexampple.DemoControllerTest.<clinit>(DemoControllerTest.java:27)
... 25 more

UPDATE I have installed docker in local (to try it out) but still I am not able to run test cases due to not able to download image. My corporate network might be an issue here. Does anyone has good solution for above situations?

Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not pull image: unexpected EOF
    at com.github.dockerjava.core.command.PullImageResultCallback.checkDockerClientPullSuccessful(PullImageResultCallback.java:96)
    at com.github.dockerjava.core.command.PullImageResultCallback.throwFirstError(PullImageResultCallback.java:111)
    at com.github.dockerjava.core.async.ResultCallbackTemplate.awaitCompletion(ResultCallbackTemplate.java:91)
    at com.github.dockerjava.core.command.PullImageResultCallback.awaitSuccess(PullImageResultCallback.java:124)

Yes, Testcontainers require a valid docker installation. When it comes to your second problem (after you've installed Docker): please try to pull the image with CLI, i.e. run a console and do `docker pull ` and post the result. — jannis, Apr 08 '20 at 21:34
Thanks @jannis. It seems my corporate VPN network does not allow to download, I tried from CLI as well and got EOF exception in that. Is there any way to get image otherway and instruct testcontainers and/or docker to work offline? — Ketan, Apr 09 '20 at 04:38
You need to get the image to the local registry somehow. Options are: a] build the image locally b] pull from remote registry (but as you wrote this does not work) c] use [docker save](https://docs.docker.com/engine/reference/commandline/save/) to export the image to a file and then [docker load](https://docs.docker.com/engine/reference/commandline/load/) to load it to local registry. — jannis, Apr 09 '20 at 14:52
how do I build the image locally? i presume I need to download somehow to my corporate machine which is stopping me to do that — Ketan, Apr 09 '20 at 18:36
`how do I build the image locally?` You need a Dockerfile and context files (any jars/scripts needed by the Dockerfile). What is this image? Is it a DB image? Spring boot application? — jannis, Apr 09 '20 at 20:45
@jannis what do you mean by valid docker container ? i have thesmae issue only in my case test container works fine locally using postgress, but when i push to gitlab my pipeline is failing, shows thesame error. — valik, Oct 08 '20 at 09:57
@valik I wrote "valid docker installation" not "valid docker container". — jannis, Oct 09 '20 at 10:32

score 86 · Answer 1 · answered Mar 19 '23 at 13:10

86

If you're using MAC, in my case, to solve the problem I had to add a link.

Follow the command below:

sudo ln -s $HOME/.docker/run/docker.sock /var/run/docker.sock

answered Mar 19 '23 at 13:10

Wanderson Xesquevixos de Sique

1,043
8
10

7

This worked for me (Mac M1) – code veda Mar 20 '23 at 12:55
5

works for M1 Pro too. – Inanc Cakil Apr 07 '23 at 12:58
1

Worked for an Intel Mac as well – gbear605 May 03 '23 at 13:48
Don't forget to restart docker after this (or at least in my case it was needed to make this solution work). – Johnczek Jun 23 '23 at 08:52
2

One addition is that mac seems to remove this ln between upgrades (I've had to bookmark this answer for that reason) – paulpdaniels Jul 20 '23 at 04:24

score 51 · Answer 2 · answered Aug 05 '21 at 09:26

51

In my case this error caused by root permission. If you are using ubuntu, docker wants permission to create container and testcontainers can not ask you for this permission at runtime. So you need to make docker runnable without sudo.

Try this steps to create docker group and add connected user:

$ sudo groupadd docker
$ sudo gpasswd -a $USER docker
$ sudo service docker restart

Try this in step 3 if you are using Ubuntu 14.04-15.10:

$ sudo service docker.io restart

After all steps try some docker command in terminal without sudo like:

$ docker ps

If you are taking permission error, restart your computer then this must be fixed.

answered Aug 05 '21 at 09:26

Emin Bilgiç

972
6
23

26

In addition to these steps, I had to ```sudo chmod 666 /var/run/docker.sock``` – Amy Doxy Mar 22 '22 at 01:26
i have this problem sudo: groupadd: command not found – Ilya Y May 24 '22 at 14:27
@IlyaY Then you are not running Ubuntu. The group management commands will differ a bit between Linux distros; look for documentation perhaps starting from `useradd` which should be reasonably standard. Another possibility is that you don't have `/sbin` on your `PATH`, which will hide this and many other administrative commands; but adding it is obviously easy, or just use the full path to the binary `/sbin/groupadd` – tripleee Aug 11 '22 at 05:48
2

@AmyDoxy `sudo chmod 666 /var/run/docker.sock` FTW!!! Thanks a lot! – ktulinho Dec 28 '22 at 16:19
remember must re-login ssh, then command will take effect – deniro.wang May 06 '23 at 12:54

score 13 · Answer 3 · answered Oct 14 '21 at 05:04

No other answers across the internet worked for me, likely because my case was somewhat unique. I am hoping this answer allows others in this weird, particular scenario to realize what is the problem.

So If:

You are on Linux
You are running your tests through an IDE
You installed IDE through flatpak, snap, or any other package manager that does sandboxing

The issue is that you are sandboxed by the package manager in the application. If you have a socket at /var/run/docker.sock (default), but the tests can't pick it up, this is why. That file does not exist in the sandbox, only on the host filesystem. Depending on the package manager, /var/run may be mounted inside the sandbox, so check your particular documentation. The most straightforward way to resolve this is to install the IDE outside of a sandbox via a tarball or some other thing.

If you're running your IDE (Intellij in my case) via Flatpak, you can add an override, like described here https://github.com/flathub/com.jetbrains.IntelliJ-IDEA-Ultimate/issues/22 via command line or in Flatseal — otocon, May 23 '22 at 11:01

score 8 · Answer 4 · answered Apr 13 '22 at 07:34

8

This issue also happens when your docker demon is not running. Starting it and running the tests again would fix the issue if it used to work before.

answered Apr 13 '22 at 07:34

Anand

693
1
8
26

score 7 · Answer 5 · edited Feb 22 '21 at 09:17

7

It could be one of several issues:

Unstable network connection as you suggested
You have not logged in to dockerhub from your terminal

In my case I needed to also add my user to the docker user group as per this link:

I also pulled the image before running the test, just in case.
One more thing I had to do was to specify the image name when initialising the TestContainers instance as is shown below:

edited Feb 22 '21 at 09:17

Dharman

30,962
25
85
135

answered Feb 22 '21 at 09:11

Joseph Waweru

1,680
18
14

Never forget to login to your dockerhub :) – JeanValjean Aug 24 '22 at 12:59
1

Lol logging in solved my issue :) Thanks. – Jamesed Jan 25 '23 at 13:46

score 3 · Answer 6 · answered Feb 13 '23 at 14:32

testcontainers tries to auto-detect how to connect to docker by inspecting the environment variable DOCKER_HOST or the file <user home>/.testcontainers.properties

In my case (Win10 + Docker Desktop on WSL2), I had to expose the docker daemon on tcp://localhost:2375 without TLS and then either: 1) set the environment variable and its value (in IDE or system) to DOCKER_HOST=tcp://localhost:2375 or 2) create the file <user home>/.testcontainers.properties and add this lines:

docker.host=tcp\://localhost\:2375
# This second line might be not completely necessary:
docker.client.strategy=org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy

BTW, if you expose the daemon, it would be better if you restrict access to it from outside your machine, using the Windows Firewall (it's not sarcasm ;-) )

score 3 · Answer 7 · answered Feb 27 '23 at 13:23

3

Maybe you forgot start docker on your computer. This solved the problem for me.

answered Feb 27 '23 at 13:23

Ярослав Федина

31
1

score 2 · Answer 8 · answered Jun 14 '23 at 11:39

None of the solutions listed here did not work for me. I installed docker desktop, and docker.sock was not in /var/run/docker.sock.

So I had to find docker.sock using ss -x -a | grep docker.sock:

u_str LISTEN 0      4096                     /home/xxx/.docker/desktop/docker.sock 37055              * 0             
u_str ESTAB  0      0                        /home/xxx/.docker/desktop/docker.sock 5890598            * 5891857       
u_str ESTAB  0      0                        /home/xxx/.docker/desktop/docker.sock 44658              * 49293

I found that the file was in $HOME/.docker/desktop/docker.sock.

So to make it work, I had to run sudo ln -s $HOME/.docker/desktop/docker.sock /var/run/docker.sock

Hope it helps you :)

I think this is due to the latest changes in Docker Desktop. Once you "Enable default Docker socket (requires password)" in Preferences -> Advanced as suggested by @Pratham the link will be created! — Jan Suchotzki, Aug 31 '23 at 12:27

score 2 · Answer 9 · answered Jun 15 '23 at 13:15

2

Testcontainers communicate to docker via sockets. In the newer version of Docker Desktop sockets are disabled by default.

This can be enabled Preferences -> Advanced -> "Enable default Docker socket (Requires password)"

answered Jun 15 '23 at 13:15

Pratham

1,522
1
18
33

score 2 · Answer 10 · answered Aug 10 '23 at 12:31

2

If you are using Docker Desktop, simply check the flag "Allow the default Docker socket to be used" in Settings

answered Aug 10 '23 at 12:31

Carlos Cuesta

1,262
1
17
20

score 1 · Answer 11 · answered Feb 24 '23 at 09:54

1

The official documentation will help you.

$ dockerd-rootless-setuptool.sh install

answered Feb 24 '23 at 09:54

Владимир Михайлов

11
1

score 0 · Answer 12 · answered Mar 29 '23 at 08:56

I had the same problem but only when I try to debug code. In my case the problem was too many breakpoints in Intellij. When I deleted all of them (especially from previous debuggings) and then put only one which I needed, the test container started normally. Maybe my response can be helpful in case when any other options don't work.

score 0 · Answer 13 · answered Apr 25 '23 at 07:50

0

In my case, Docker Desktop wasn't able to pull images because the CPU usage was 100%. Restarting the PC fixed my issue,

answered Apr 25 '23 at 07:50

mnagdev

384
3
11

score 0 · Answer 14 · answered Aug 10 '23 at 03:14

I had a similar exception on my M1 Mac, when switching from Docker Desktop to Rancher Desktop. The approach suggested by @Wanderson Xesquevixos de Sique turned out to solve things, except that the path for symbolic linkage turned out to be different:

sudo ln -s ~/.rd/docker.sock /var/run/docker.sock

Test Container test cases are failing due to "Could not find a valid Docker environment"

14 Answers14

Linked