Cannot list out stopped container using docker ps -a

Question

I tried to remove a image using docker rmi image_id and terminal output Error response from daemon: conflict: unable to delete dd85c346890b (must be forced) - image is being used by stopped container b91e905c2b8e. Then I tried to list out the container using docker ps -a but it does not shown. Then I tried docker inspect b91e905c2b8e it does exists. And I can remove the container using docker rm b91e905c2b8e.

Why the container is not shown using docker ps -a. The container is created using kubectl apply pod_config.yaml. Will this affect docker?

Edited:
pod.yaml

apiVersion: v1 
kind: Pod 
metadata: 
  name: my-pod
  labels: 
    component: web
spec: 
  containers: 
    - name: client 
      image: imageB // original is imageA    
      ports: 
        - containerPort: 3000

Check if the pod was scheduled on the node where you are doing docker commands — Arghya Sadhu, Aug 08 '20 at 13:47
You probably shouldn't manually use `docker` commands on a node managed by Kubernetes; delete the Kubernetes Pod (or its controlling Deployment/StatefulSet/Job/DaemonSet) instead. If you're not in a purely-local environment (Docker Desktop) you typically shouldn't/can't log into the worker nodes at all. — David Maze, Aug 08 '20 at 14:18
the pod previously use the container (b91e905c2b8e), but now replaced by a new another container and is running. Can you explain what is happening?@ArghyaSadhu — HKIT, Aug 08 '20 at 14:19
Because I changed the container inside the pod from A to B, but A remains, also, its image. How can I remove them? I cannot remove the pod as the pod is still in use. @DavidMaze — HKIT, Aug 08 '20 at 14:21
If the pod is still in use, then the image is still in use, and you can't delete it. — David Maze, Aug 08 '20 at 14:26
No, actually I changed the image in the Pod, like from image A to image B in the pod yaml, so I want to remove image A and its container now? @DavidMaze — HKIT, Aug 08 '20 at 14:27
Please share more info about your k8s cluster as we don't know anything about your installation. Is it one-node cluster or does it have more nodes ? If it has more nodes, on which node are you running your docker commands ? — mario, Aug 10 '20 at 10:17
@mario Hi Mario, I am using Kubernetes in Docker for mac, it is single node cluster. I created the pod.yaml apply it using kubectl. Then I change the image, from imageA to imageB, and apply using kubectl again. — HKIT, Aug 10 '20 at 14:20
Do you push your images to some registry before using them ? Or maybe you build them and store them only locally ? Are you sure `Image B` isn't somehow dependent on `Image A` ? Isn't it using same layers ? If `image A` isn't used any more by any container it should be possible to remove it without any problem. If it cannot be removed, it is probably still in use. — mario, Aug 11 '20 at 13:45
@mario I pushed them to docker hub. They are 2 totally independent image. The message is "image is being used by stopped container b91e905c2b8e". But my question is "Why the stopped container is not shown using docker ps -a"? — HKIT, Aug 11 '20 at 13:57
Hmm... this is quite an interesting question as normally such scenario shouldn't be possible. — mario, Aug 11 '20 at 14:07
Where are you actually running `docker ps -a` ? On your Mac ? Or you first attach to your k8s node VM using [screen](https://stackoverflow.com/questions/39739560/how-to-access-the-vm-created-by-dockers-hyperkit/39747831#39747831) ? — mario, Aug 26 '20 at 13:40
Could you additionally share your **Docker for Desktop** version and **Docker CLI** version ? — mario, Aug 26 '20 at 13:43
@mario I run docker ps -a on my mac, not using screen. Below is the version Docker for Desktop: 2.3.0.4 Docker Cli: 19.03.12 I am wondering is there any scope concept inside docker? — HKIT, Aug 27 '20 at 02:57

VAS · Answer 1 · 2020-08-28T13:57:56.113

Quick solution:

docker --host unix:///$HOME/Library/Containers/com.docker.docker/Data/docker.raw.sock ps -a

Explanation:

It seems to be an expected behavior from Docker developers point of view.
Mac version of Docker is using specific socket to connect docker-cli to the daemon that runs on Hyperkit VM.

It works in exactly the same way if you are using docker binary shipped with Docker.app or installed separately.

Luckily Docker-desktop for MacOS creates two kind of sockets:

A "filtered" socket ( used by default ) for the purpose of Docker-desktop application requirements, probably related to its Kubernetes implementation:

$HOME/Library/Containers/com.docker.docker/Data/docker.sock

and the unfiltered one:

$HOME/Library/Containers/com.docker.docker/Data/docker.raw.sock

To show the differences, I've reproduced your situation and got the container with ID: b2f1fed9c793 in Exited state:

$ docker inspect b2f1fed9c793 | grep Status
            "Status": "exited",

$ docker  ps -a | grep b2f1fed9c793
# empty output

When I run the same command connecting to the different sockets I'm getting different results and only raw socket shows exited containers:

$ docker --host unix:///$HOME/Library/Containers/com.docker.docker/Data/docker.sock ps -a | grep b2f1fed9c793
# empty output

$ docker --host unix:///$HOME/Library/Containers/com.docker.docker/Data/docker.raw.sock ps -a | grep b2f1fed9c793
b2f1fed9c793        nginx                              "nginx -g 'daemon of…"   40 minutes ago      Exited (0) 39 minutes ago                            k8s_nginx_nginx3_default_2a5ed89d-ccff-4561-a320-676603d30619_4

Default socket gives 0 lines with Exited containers:

$ docker --host unix:///$HOME/Library/Containers/com.docker.docker/Data/docker.sock ps -a | grep Exited | wc -l 
       0

But raw socket gives 22:

$ docker --host unix:///$HOME/Library/Containers/com.docker.docker/Data/docker.raw.sock ps -a | grep Exited | wc -l 
      22

To use the raw socked by default you can create an alias:

$ alias docker="docker --host unix:///$HOME/Library/Containers/com.docker.docker/Data/docker.raw.sock"

$ docker  ps -a | grep Exited | wc -l 
      22

The solution was originally suggested by lalyos in the issue comment, but he was solving slightly different problem.

Cannot list out stopped container using docker ps -a

1 Answers1