2

Any idea why I can't run docker using Airflow?

The same docker command runs fine from terminal using the user that runs airflow. So there is no permission issue on the Linux side.

But when put it into Airflow dag, it complains

docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.39/containers/create: dial unix /var/run/docker.sock: connect: permission denied.

The command is simply docker run --rm -v /data:/data:ro docker_image mycommand. It is wrapped inside a python subprocess, so I can't use docker operator.

Note this is not the same issue with the question: How to fix "dial unix /var/run/docker.sock: connect: permission denied" when group permissions seem correct? There it is a linux permission issue as it cannot run the docker run command. Here the problem is more with Airflow, I think.

Luke
  • 720
  • 1
  • 9
  • 22
  • Can you add the code snippet for this and also for the successful condition – Avinash Reddy Jun 27 '19 at 03:11
  • Possible duplicate of [How to fix "dial unix /var/run/docker.sock: connect: permission denied" when group permissions seem correct?](https://stackoverflow.com/questions/51342810/how-to-fix-dial-unix-var-run-docker-sock-connect-permission-denied-when-gro) – mchawre Jun 27 '19 at 04:08
  • @mchawre The mentioned question is a linux permission problem. The issue I have is with Apache Airflow. – Luke Jun 27 '19 at 13:38
  • The user that Airflow is running as doesn't have permission to access `/var/run/docker.sock`. There are several ways of working around this, but I'd recommend Impersonating a more privileged user using `run_as_user` on the Airflow action that runs docker: https://airflow.apache.org/security.html#impersonation – Art Gillespie Jul 02 '19 at 13:53
  • @ArtGillespie The user that Airflow is running has the privilege to run docker command. There is no permission issue on Linux side. – Luke Jul 02 '19 at 16:52
  • This worked for me and its clean too, [link](https://stackoverflow.com/questions/57724705/windows-docker-permission-denied-var-run-docker-sock?rq=1) – Glitch Jan 13 '21 at 06:31

2 Answers2

2

Faced a similar issue . Got it fixed by changing the default_owner(default=airflow) in airflow.cfg to the user having permission to access docker(i.e username belonging to docker group).

Interestingly passing owner in 'default_args' in the DAG definition didn't seem to work

Nixon Raphy
  • 312
  • 2
  • 20
0

Looks like the reason is in access rights to /var/run/docker.sock. For dev environment, you can do:

docker exec -ti -u root <container> bash
sudo chmod 777 /var/run/docker.sock
Bogdan Dubas
  • 9
  • 1