86

Issue: Can not stop docker containers, whenever I try to stop containers I get the following Error message,

ERROR: for yattyadocker_web_1  cannot stop container: 1f04148910c5bac38983e6beb3f6da4c8be3f46ceeccdc8d7de0da9d2d76edd8: Cannot kill container 1f04148910c5bac38983e6beb3f6da4c8be3f46ceeccdc8d7de0da9d2d76edd8: rpc error: code = PermissionDenied desc = permission denied

OS Version/build: Ubuntu 16.04 | Docker Version 17.09.0-ce, build afdb6d4 | Docker Compose version 1.17.1, build 6d101fb

Steps to reproduce:

  • Created a rails project with Dockerfile and docker-compose.yml. docker-compose.yml is of version 3.
  • Image is built successfully with either docker build -t <project name> . or docker-compose up --build
  • Containers boots up and runs successfully.
  • Try to stop docker compose with docker-compose down.

What I tried::

  • I have to run sudo service docker restart and then the containers can be removed.
  • Uninstalled docker, removed docker directory and then re installed everything. Still facing same issue.

Note: This configuration was working correctly earlier, but somehow file permissions might have changed and I am seeing this error. I have to run sudo service docker restart and then the containers can be removed. But this is highly inconvenient and I don't know how to troubleshoot this.

Reference Files:

# docker-compose.yml
version: '3'
volumes:
  db-data:
    driver: local
  redis-data:
    driver: local  
services:
  db:
    image: postgres:9.4.1
    volumes:
      - db-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    env_file: local_envs.env
  web:
    image: yattya_docker:latest
    command: bundle exec puma -C config/puma.rb
    tty: true
    stdin_open: true
    ports:
      - "3000:3000"
    links:
      - db
      - redis
      - memcached
    depends_on:
      - db
      - redis
      - memcached
    env_file: local_envs.env
  redis:
    image: redis:3.2.4-alpine
    ports:
      # We'll bind our host's port 6379 to redis's port 6379, so we can use
      # Redis Desktop Manager (or other tools) with it:
      - 6379:6379
    volumes:
      # We'll mount the 'redis-data' volume into the location redis stores it's data:
      - redis-data:/var/lib/redis
    command: redis-server --appendonly yes
  memcached:
    image: memcached:1.5-alpine
    ports:
      - "11211:11211"
  clock:
    image: yattya_docker:latest
    command: bundle exec clockwork lib/clock.rb
    links:
      - db
    depends_on:
      - db
    env_file: local_envs.env
  worker:
    image: yattya_docker:latest
    command: bundle exec rake jobs:work
    links: 
      - db
    depends_on: 
      - db
    env_file: local_envs.env

And Dockerfile:

# Dockerfile
FROM ruby:2.4.1

RUN apt-get update && apt-get install -y nodejs --no-install-recommends && rm -rf /var/lib/apt/lists/*

ENV APP_HOME /app
RUN mkdir -p $APP_HOME
WORKDIR $APP_HOME

ADD Gemfile* $APP_HOME/
RUN bundle install

ADD . $APP_HOME

RUN mkdir -p ${APP_HOME}/log
RUN cat /dev/null > "$APP_HOME/log/development.log"

RUN mkdir -p ${APP_HOME}/tmp/cache \
    && mkdir -p ${APP_HOME}/tmp/pids \
    && mkdir -p ${APP_HOME}/tmp/sockets

EXPOSE 3000
Parth Modi
  • 1,675
  • 2
  • 20
  • 26
  • Which user does start the containers? – C. Hediger Nov 10 '17 at 12:59
  • I start containers on my ubuntu.. docker user created while installing docker must be used for managing containers. I am new to docker. – Parth Modi Nov 10 '17 at 13:06
  • 2
    Did you add your user account to the docker group? `sudo usermod -aG docker ${USER}` this will allow you to work with docker without sudo – Shawn C. Nov 10 '17 at 13:59
  • @ShawnC is right (at least I suspect). To validate you can run "sudo docker compose" or "sudo docker-compose down". – mikey Nov 10 '17 at 14:41
  • Yes.. I followed the guide and added docker to group, I can run docker commands without sudo. I am not having problem with `docker-compose up`, it's just causing trouble while removing or stopping containers. – Parth Modi Nov 10 '17 at 16:47
  • I'm experiencing the same issue. Ubuntu 10.04, Docker version 17.09.0-ce, build afdb6d4, docker-compose version 1.8.0. Tried re-installing docker-ce with `apt-get prune` and `apt-get autoremove`. No luck. Restarting docker service with `sudo service docker restart` works. – Charlie Vieillard Nov 16 '17 at 10:17
  • 1
    @CharlieVieillard If you have properly set `sudo usermod -aG docker ${USER}` and still facing issue then You might need to check whether apparmor is working fine or not. I had issues with apparmor, hence I re-installed docker and apparmor. And after **restarting laptop** the issue was resolved. I think issue is related to https://github.com/moby/moby/issues/20554 – Parth Modi Nov 16 '17 at 11:51
  • @ParthModi Thanks, removing apparmor worked! – Charlie Vieillard Nov 16 '17 at 14:04

8 Answers8

236

I installed Docker from the snap package and after a while I decided to move to apt repository installation.

I was facing the same problem and using sudo aa-remove-unknown worked for me.

So no reinstallation of Apparmor was needed.

Alejandro S.
  • 2,479
  • 2
  • 6
  • 6
  • 17
    This is the correct non-nuclear answer. Installing Docker as a `snap` adds a bunch of AppArmor profiles, that conflict with and `apt` installation. This happened to me by accident, as I was looking for a convenient method of installing everything Docker. Opted for an `apt` installation in the end, as it gave me more control and is more up-to-date. – sesodesa May 15 '19 at 15:59
  • 6
    Perfect answer !! – NEERAJ SWARNKAR May 17 '20 at 10:59
  • using ```sudo aa-remove-unknown``` causing my mysql workbench does not work and shows error: Failed to Query AppArmor Policy: No such file or directory. edit: all apps installed with snap don't work – theboyonfire Jan 21 '21 at 11:18
  • I restarted my computer and the apps installed with snap has gone .. any idea ? – theboyonfire Jan 21 '21 at 11:54
  • Still working through this, but for me, this did indeed appear to cause problems with at least some snaps. – codenoob Apr 07 '21 at 23:13
  • 1
    I was able to get my snaps back working with: "sudo apparmor_parser -r /var/lib/snapd/apparmor/profiles/*" Thanks to jdstrand, https://forum.snapcraft.io/t/snap-confine-has-elevated-permissions-and-is-not-confined-but-should-be-refusing-to-continue-to-avoid-permission-escalation-attacks/5557/2 – codenoob Apr 08 '21 at 00:20
  • This seemed to work, however, after a reboot, the problem was back. – not2savvy Sep 13 '21 at 13:29
  • I got the same problem like @not2savvy . The problem is back after restart. Have anyone solved this one completely ? – ninoorta Jun 05 '22 at 10:49
  • @ninoorta I was able to solve it for me. Please [see my answer here](https://stackoverflow.com/a/68132254/4850949). – not2savvy Jun 05 '22 at 13:46
  • Plus https://stackoverflow.com/a/54088663/4850949 AFAIR. – not2savvy Jun 05 '22 at 13:54
  • I'm a noob. For future people trying to use sudo aa-remove-unknown be careful how you use it because it ended removing some libraries necessary for other programs – Mangostino Nov 17 '22 at 21:52
  • Apps installed with the snap will not start after that – iku Jan 02 '23 at 16:54
115

For anyone that does not wish to completely purge AppArmor.

Check status: sudo aa-status

Shutdown and prevent it from restarting: sudo systemctl disable apparmor.service --now

Unload AppArmor profiles: sudo service apparmor teardown

Check status: sudo aa-status

You should now be able to stop/kill containers.

Justin Emery
  • 1,644
  • 18
  • 25
jsloan117
  • 1,159
  • 2
  • 7
  • 2
  • 12
    Attention! Some apps won't start after that – Vadim Filin Apr 23 '19 at 15:16
  • 2
    Be aware, any solution containing `apparmor` will break snap and all the packages installed via snap and even after that I still have problem with running and killing pods on kubernetes. – AVarf Jul 19 '19 at 12:04
  • I followed the steps described above. Now some of my apps don't start anymore. How can I fix that? – HerthaBSC Oct 29 '19 at 12:09
  • @HerthaBSC I have the same issue. IntelliJ stopped working. I managed to temporary fix it. Using `sudo apt purge snapd && sudo apt install snapd` solved the issue. But it I guess it purges the apps that weren't starting. IntelliJ, in my case. So, I had to install it again. But once you reboot the system, you have to repeat the process. Did you manage to find a permanent solution? – Jim Jan 23 '20 at 14:06
  • 2
    If I remember it right, I disabled AppArmor as described in the solution above, then reinstalled Docker and reenabled AppArmor afterwards with `sudo systemctl enabled apparmor` and `sudo systemctl start apparmor`. – HerthaBSC Jan 25 '20 at 08:48
33

A direct fix to the problem is executing bash in the container to be killed and directly calling kill there. An example:

host$ docker exec -it <container-name> sh
container$ ps
PID   USER     TIME  COMMAND
    1 root      0:00 {entrypoint.sh} /bin/sh /entrypoint.sh
   16 root      0:00 {entrypoint.sh} /bin/sh /entrypoint.sh
   24 root      0:00 sh
   31 root      0:00 ps
container$ kill 1

To check that the container was killed, run docker ps. This is a useful alternative to the solution reinstalling apparmor as this will also remove snapd.

Moritz
  • 2,987
  • 3
  • 21
  • 34
  • 2
    this does not work, it just list `sh` and `ps` processes in my cases (from a docker-compose file) – Pak May 09 '19 at 16:00
  • 3
    I can confirm that this works. I also believe that this should be the accepted answer as this doesn't have any risks for your host system. – Twoez Sep 15 '22 at 08:39
  • We had a different situation where ps showed that process 1 was running bash. We had to kill -9 1 instead of just kill 1. That worked to stop the container. – Prof Von Lemongargle Feb 27 '23 at 15:07
  • I have tried to run "kill -9 1", where "1" is the process id, but it stays. What else can I do ? – user3595231 Jun 27 '23 at 00:26
14

I was able to fix the issue. Apparmor service in ubuntu was not working normally due to some unknown issue. The problem was similar to the issue reported in moby project https://github.com/moby/moby/issues/20554.

The /etc/apparmor.d/tunables folder was empty, and https://github.com/mlaventure suggested to purge/reinstall apparmor to get it to the initial state.

So I reinstalled apparmor, and after restarting the problem was solved.

Hope this helps.

Parth Modi
  • 1,675
  • 2
  • 20
  • 26
  • 1
    Further details on docker hub forum: https://forums.docker.com/t/can-not-stop-docker-container-permission-denied-error/41142/5 – James Feb 25 '18 at 20:53
12

In my case the issue was that I had conflicting docker installations: docker itself from the official docker-ce package , but docker-compose from the Ubuntu snap package.

Installing correctly docker-compose from the official github (instructions here) did the trick. I also followed the Linux post-install instructions and it may have helped as well (to run docker as a non-root user)

I just left AppArmor alone here - I did not touch it.

OlivierBondu
  • 177
  • 2
  • 8
  • 6
    Had the same issue. I saw that Docker was installed through snap: `snap list` and used `sudo snap remove docker` to remove it first – Zenuka Feb 06 '21 at 08:21
0

OS: Ubuntu 22.04 LTS docker version: 20.10.17, build 100c701 docker-compose version: 1.29.2

I faced the same issue and tried following,

  1. sudo systemctl disable apparmor.service --now
  2. sudo service apparmor teardown
  3. sudo aa-remove-unknown
  4. reboot

These solutions didn't work for me. This issue happens because of a security feature of linux kernal, apparmor.

We can disable it by running docker daemon as a non-root user(Rootless mode), Execute following commands,

Solution:

  1. curl -fsSL https://get.docker.com/rootless | sh | FORCE_ROOTLESS_INSTALL=1
  2. export PATH=/home/user-directory/bin:$PATH

docker-compose down or docker rm, will work

0

I had trouble with this for so long so first I realized I had to terminate the network that the container was one. So I followed all the steps for that. But I was still getting permission denied. Then just did

sudo docker inspect portainer

And in the "State" -> "Pid", I then used the Pid with

sudo kill

0

it just simple resolves work for me

 sudo snap remove docker
Mayur Dagdi
  • 11
  • 1
  • 4