457

Despite Docker's Interactive tutorial and faq I lose my data when the container exits.

I have installed Docker as described here: http://docs.docker.io/en/latest/installation/ubuntulinux without any problem on ubuntu 13.04.

But it loses all data when exits.

iman@test:~$ sudo docker version
Client version: 0.6.4 
Go version (client): go1.1.2 
Git commit (client): 2f74b1c 
Server version: 0.6.4 
Git commit (server): 2f74b1c 
Go version (server): go1.1.2 
Last stable version: 0.6.4 


iman@test:~$ sudo docker run ubuntu ping
2013/10/25 08:05:47 Unable to locate ping 
iman@test:~$ sudo docker run ubuntu apt-get install ping
Reading package lists... 
Building dependency tree... 
The following NEW packages will be installed: 
  iputils-ping 
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. 
Need to get 56.1 kB of archives. 
After this operation, 143 kB of additional disk space will be used. 
Get:1 http://archive.ubuntu.com/ubuntu/ precise/main iputils-ping amd64 3:20101006-1ubuntu1 [56.1 kB] 
debconf: delaying package configuration, since apt-utils is not installed 
Fetched 56.1 kB in 0s (195 kB/s) 
Selecting previously unselected package iputils-ping. 
(Reading database ... 7545 files and directories currently installed.) 
Unpacking iputils-ping (from .../iputils-ping_3%3a20101006-1ubuntu1_amd64.deb) ... 
Setting up iputils-ping (3:20101006-1ubuntu1) ... 
iman@test:~$ sudo docker run ubuntu ping
2013/10/25 08:06:11 Unable to locate ping 
iman@test:~$ sudo docker run ubuntu touch /home/test
iman@test:~$ sudo docker run ubuntu ls /home/test
ls: cannot access /home/test: No such file or directory

I also tested it with interactive sessions with the same result. Did I forget something?

EDIT: IMPORTANT FOR NEW DOCKER USERS

As @mohammed-noureldin and others said, actually this is NOT a container exiting. Every time it just creates a new container.

iman
  • 21,202
  • 8
  • 32
  • 31
  • 13
    That cannot be called "**container exiting**", you are just creating a new container, using exiting word can confuse a lot (I was confused because of that too). – Mohammed Noureldin Dec 28 '16 at 15:06
  • 2
    @MohammedNoureldin, you are right, exiting is not correct, but this is exactly what you, I and others thought. So it's a better word in question, your edit makes question an answer! New searchers will not find here! – iman Dec 30 '16 at 09:18
  • At my very beginning point with Docker, I thought that actually because of YOUR question, I find that address is simply WRONG. the new title was reviewed and accepted, I don't understand why should some one insist on a wrong title, it is your quesion, and your descision. – Mohammed Noureldin Dec 30 '16 at 14:33
  • 3
    I agree with @MohammedNoureldin. The combination of the specific title, example and accepted answer does not help future readers and especially beginners understand `Docker`. I would propose to keep the title and the original question, since beginners will definitely search for something like this. But, why don't you add something describing your misconceptions at the time you wrote the post. It will help make things clear. This is our culture here at SO... isn't it? :-) – tgogos Jun 24 '17 at 20:56
  • 2
    I had this problem... every time u exit u have to start your container not run it... run an image agane make a new container this will help docker start docker attach – fatemeh Jul 28 '18 at 06:22

11 Answers11

449

You need to commit the changes you make to the container and then run it. Try this:

sudo docker pull ubuntu

sudo docker run ubuntu apt-get install -y ping

Then get the container id using this command:

sudo docker ps -l

Commit changes to the container:

sudo docker commit <container_id> iman/ping

Then run the container:

sudo docker run iman/ping ping www.google.com

This should work.

JamesThomasMoon
  • 6,169
  • 7
  • 37
  • 63
Unferth
  • 4,638
  • 1
  • 14
  • 14
  • 16
    So I should use *commit* after each *run* to preserve data. – iman Oct 25 '13 at 10:53
  • 9
    Commit should be used only when you make CHANGES in the container(like installing new tools or data) so that those changes are saved and the next time when you run a new container from that Image, it will start from the point of last save or commit, preserving your data. – Unferth Oct 25 '13 at 11:39
  • 8
    @Unferth what if I want to keep committing changes? So far it creates more images with ``. How do I keep appending the commit on top of an existing image? – Marconi Mar 02 '14 at 04:15
  • 2
    Does it mean, whenever you test some code in Daemon and want to take a look at the output after it finished. You have to commit and `docker run -t -i /bin/bash`? to log in back to the container to view the output? – B.Mr.W. Jul 04 '14 at 18:26
  • 69
    Incrementally committing changes is not "the docker way". Use a DOCKERFILE. – user2105103 Jan 14 '15 at 14:23
  • 31
    How would you commit from within the container? Consider the following scenario: 1) I am running the container like this: docker run -i -t myimage /bin/bash 2) I make some changes 3) I cannot commit from within the container, so when I exit the container, I will loose all my data, without having the chance to commit my previous changes – qgicup Jun 18 '15 at 15:15
  • @qgicup basically, you can't commit from the container. The only way do that would be very convoluted and not recommended for a typical app (that is, one way would be to have docker available in the image and mount the socket, which you're unlikely to be wanting outside of very specific use cases). And btw, your data is actually not lost. The right way to handle data is to use _volumes_/_data containers_. Read the official docs about that, it's well documented. There's also a video tutorial from botchagalupe. HTH – Baptiste Mathus Sep 08 '15 at 08:52
  • Note that when you use docker-machine you don't need to use sudo. – skywalker Apr 29 '16 at 13:48
  • @user2105103 The reason to use `docker run` and then `docker commit` is because `docker build` does not have a `--privileged` mode. `docker run` is the only docker command with `--privileged`. Some processes run during docker creation will require `--privileged`. See [docker #1916](https://github.com/docker/docker/issues/1916) – JamesThomasMoon Aug 10 '16 at 18:53
  • 2
    @qgicup I believe Ctrl+p, Ctrl+q will exit the current container while keeping it running. – schumacher574 Dec 05 '17 at 18:04
  • Awesome answer, this is exactly what I was looking for. The key to this is the `-l` argument which shows the last modified container. – domdambrogia Feb 07 '18 at 23:51
  • @schumacher574 What does *keeping it running* mean? Do you mean *it will save the current state but release the memory/cpu resources until restarted* ? – WestCoastProjects Feb 20 '18 at 20:02
  • 2
    @javadba No. AFAIK, Ctrl+p, Ctrl+q detaches the tty without exiting the shell. So the processes within the container continue to run. I mentioned it in response to qgicup's scenario posed in his comment as a way to detach from the container's tty and have a chance to commit or execute whatever other command is necessary. – schumacher574 Feb 20 '18 at 21:58
  • Better use `docker ps -a`. For me `docker ps -l` showed stopped version of an image, which I mistakengly commited at that point. The actual "running" image was only shown by `docker ps -a`. – Hi-Angel Aug 27 '19 at 12:40
425

When you use docker run to start a container, it actually creates a new container based on the image you have specified.

Besides the other useful answers here, note that you can restart an existing container after it exited and your changes are still there.

docker start f357e2faab77 # restart it in the background
docker attach f357e2faab77 # reattach the terminal & stdin
Robin Green
  • 32,079
  • 16
  • 104
  • 187
ZeissS
  • 11,867
  • 4
  • 35
  • 50
  • 113
    `docker ps` shows you only running docker containers. `docker ps -a` shows you also the ones that have exited -- and that you can keep running. A commit is only necessary after each run if you want to make a snapshot there for future use, otherwise the container itself will stick around for you to keep using. – user1278519 Apr 29 '14 at 16:58
  • 2
    Question please, so if i download a `jenkins` server docker and i run it on my ci host and it runs some jobs i have and as a result jenkins server writes down to disk some logs. now if my server (which hosted my docker) is restarted and I start again my jenkins docker would it mean I lost all the log files? if that's so how can I possibly use `jenkins` docker for example to ease my jenkins installation on CI? – Jas Feb 23 '15 at 07:27
  • 2
    @Jas If you stick to the same container and not create new ones, there is no problems. Docker nowadays has restart policies, so you can configure it to restart the same container on machine reboot. I would also advise you to put your jenkins home into a volume to be able to access it from the outside (backups etc). – ZeissS Feb 23 '15 at 11:33
  • 7
    Here's a handy way to copy files out of the container once you exit: `docker cp $(docker ps -alq):/path/to/file .` – vhs Jun 03 '15 at 15:31
  • 3
    you can also start and attach a container by it's name. (e.g. `docker run -it --name my_debian debian` and after `docker start my_debian && docker attach my_debian`) – Johnny Willer Jun 18 '17 at 01:43
  • Quick question: If I don't commit, I lost the data. It's clear. But when I change nginx configuration, why it remains updated ? (no need to commit). I have hadoop in the docker - so just don't know when to commit not to lose data (For example no need to commit when I upload new files etc). Just don't understand how docker decides what to save and what not to save. – grep Jul 16 '19 at 21:02
149

There are following ways to persist container data:

  1. Docker volumes

  2. Docker commit

    a) create container from ubuntu image and run a bash terminal.

       $ docker run -i -t ubuntu:14.04 /bin/bash

    b) Inside the terminal install curl

       # apt-get update
   # apt-get install curl

    c) Exit the container terminal

       # exit

    d) Take a note of your container id by executing following command :

       $ docker ps -a

    e) save container as new image

       $ docker commit <container_id> new_image_name:tag_name(optional)

    f) verify that you can see your new image with curl installed.

       $ docker images           

   $ docker run -it new_image_name:tag_name bash
      # which curl
        /usr/bin/curl
kalyani chaudhari
  • 7,515
  • 3
  • 24
  • 21
  • Is it necessary to `exit` before `docker commit`? Thanks. – Abhishek Anand Aug 23 '18 at 15:22
  • 2
    @AbhishekAnand yes, because with the `docker run` command you run bash in the container and you stay in there due to the `-i` and `-t` options (interactive with TTY). However, Docker runs on you machine, outside of the container, so after making the necessary changes to the container from the inside, to go back to your system's shell you have to `exit` (or Ctrl+D) the container's shell. Also note the `#` and `$` in the answer, which indicate the different shells the commands are written to. – Erik Sep 30 '18 at 08:52
  • Quick question: If I don't commit, I lost the data. It's clear. But when I change nginx configuration, why it remains updated ? (no need to commit) @Erik – grep Jul 16 '19 at 21:01
  • @grep if you have a clear and reproducible MWE, please ask new question about it, if there isn't one already about this specific use case. – Erik Jul 31 '19 at 17:22
  • 3. `docker stop` followed by `docker start`. – carillonator Sep 27 '19 at 13:40
  • Notice that you can update the same image if you name it with the old name without creating a new one each time – bormat Dec 09 '19 at 16:31
  • @kalyani chaudhari: Do I need to commit my image everytime I install something new? – Amir Sep 06 '20 at 17:59
62

In addition to Unferth's answer, it is recommended to create a Dockerfile.

In an empty directory, create a file called "Dockerfile" with the following contents.

FROM ubuntu
RUN apt-get install ping
ENTRYPOINT ["ping"]

Create an image using the Dockerfile. Let's use a tag so we don't need to remember the hexadecimal image number.

$ docker build -t iman/ping .

And then run the image in a container.

$ docker run iman/ping stackoverflow.com
Aayush Pathak
  • 159
  • 11
salathe
  • 51,324
  • 12
  • 104
  • 132
  • 1
    Never having to do it manually more than once is exactly the point of docker. Create a dockerfile, commit and upload the resulting image. Pull said image going forward. – Brandon Bertelsen Nov 16 '18 at 07:10
18

There are really great answers above to the asked question. There might be no need for another answer but still I want to give my personal opinion on the topic in the simplest words possible.

Here are some points about containers & images that will help us for a conclusion:

  • A docker image can be:
    1. created-from-a-given-container
    2. deleted
    3. used-to-create-any-number-of-containers
  • A docker container can be:
    1. created-from-an-image
    2. started
    3. stopped
    4. restarted
    5. deleted
    6. used-to-create-any-number-of-images
  • A docker run command does this:
    1. Downloads an image or uses a cached image
    2. Creates a new container out of it
    3. Starts the container
  • When a Dockerfile is used to create an image:
    1. It is already well known that the image will eventually be used to run a docker container.
    2. After issuing docker build command, docker behind-the-scenes creates a running container with a base-file-system and follows steps inside the Dockerfile to configure that container as per the developers need.
    3. After the container is configured with specs of the Dockerfile, it will be committed as an image.
    4. The image gets ready to rock & roll!

Conclusion:

As we can see, a docker container is independent of a docker image.

A container can be restarted provided the unique ID of that container [use docker ps --all to get the id].

Any operation like making a new directory, creating files, installing tools, etc. can be done inside the container when it is running. Once the container is stopped, it persists all the changes. Container stopping and restarting is like rebooting a computer system.

An already created container is always available for a restart but when we issue docker run command, a new container is created out of an image and hence it is like a new computer system. The changes made inside the old container - as we can understand now - are not available in this new container.

A final note:

I guess it's now obvious why the data seems to be lost yet it is always there.. but in a different [old] container. So, take a good note of the difference in docker start & docker run command & never get confused in them.

bluelights
  • 1,086
  • 8
  • 12
16

I have got a much simpler answer to your question, run the following two commands

sudo docker run -t -d ubuntu --name mycontainername /bin/bash
sudo docker ps -a

the above ps -a command returns a list of all containers. Take the name of the container which references the image name - 'ubuntu' . docker auto generates names for the containers for example - 'lightlyxuyzx', that's if you don't use the --name option.

The -t and -d options are important, the created container is detached and can be reattached as given below with the -t option.

With --name option, you can name your container in my case 'mycontainername'.

sudo docker exec -ti mycontainername bash

and this above command helps you login to the container with bash shell. From this point on any changes you make in the container is automatically saved by docker. For example - apt-get install curl inside the container You can exit the container without any issues, docker auto saves the changes.

On the next usage, All you have to do is, run these two commands every time you want to work with this container.

This Below command will start the stopped container:

sudo docker start mycontainername

sudo docker exec -ti mycontainername bash

Another example with ports and a shared space given below:

docker run -t -d --name mycontainername -p 5000:5000 -v ~/PROJECTS/SPACE:/PROJECTSPACE 7efe2989e877 /bin/bash

In my case: 7efe2989e877 - is the imageid of a previous container running which I obtained using

docker ps -a

Magnus Melwin
  • 1,509
  • 1
  • 21
  • 32
  • 5
    With Docker 18.09.2 on Ubuntu 18.04 it doesn't work as is. It works if I put the `--name` and option before the image name, like this: `docker run --name mycontainername -t -d ubuntu /bin/bash` – Stéphane Gourichon Apr 14 '19 at 20:06
8

You might want to look at docker volumes if you you want to persist the data in your container. Visit https://docs.docker.com/engine/tutorials/dockervolumes/. The docker documentation is a very good place to start

Scott Stensland
  • 26,870
  • 12
  • 93
  • 104
Kudzanayi Mutamba
  • 89
  • 1
  • 4
4

My suggestion is to manage docker, with docker compose. Is an easy to way to manage all the docker's containers for your project, you can map the versions and link different containers to work together.

The docs are very simple to understand, better than docker's docs.

Docker-Compose Docs

Best

Cam T
  • 1,965
  • 2
  • 8
  • 8
1

the similar problem (and no way Dockerfile alone could fix it) brought me to this page.

stage 0: for all, hoping Dockerfile could fix it: until --dns and --dns-search will appear in Dockerfile support - there is no way to integrate intranet based resources into.

stage 1: after building image using Dockerfile (by the way it's a serious glitch Dockerfile must be in the current folder), having an image to deploy what's intranet based, by running docker run script. example: docker run -d \ --dns=${DNSLOCAL} \ --dns=${DNSGLOBAL} \ --dns-search=intranet \ -t pack/bsp \ --name packbsp-cont \ bash -c " \ wget -r --no-parent http://intranet/intranet-content.tar.gz \ tar -xvf intranet-content.tar.gz \ sudo -u ${USERNAME} bash --norc"

stage 2: applying docker run script in daemon mode providing local dns records to have ability to download and deploy local stuff.

important point: run script should be ending with something like /usr/bin/sudo -u ${USERNAME} bash --norc to keep container running even after the installation scripts finishes.

no, it's not possible to run container in interactive mode for the full automation matter as it will remain inside internal shall command prompt until CTRL-p CTRL-q being pressed.

no, if interacting bash will not be executed at the end of the installation script, the container will terminate immediately after finishes script execution, loosing all installation results.

stage 3: container is still running in background but it's unclear whether container has ended installation procedure or not yet. using following block to determine execution procedure finishes: while ! docker container top ${CONTNAME} | grep "00[[:space:]]\{12\}bash \--norc" - do echo "." sleep 5 done the script will proceed further only after completed installation. and this is the right moment to call: commit, providing current container id as well as destination image name (it may be the same as on the build/run procedure but appended with the local installation purposes tag. example: docker commit containerID pack/bsp:toolchained. see this link on how to get proper containerID

stage 4: container has been updated with the local installs as well as it has been committed into newly assigned image (the one having purposes tag added). it's safe now to stop container running. example: docker stop packbsp-cont

stage5: any moment the container with local installs require to run, start it with the image previously saved. example: docker run -d -t pack/bsp:toolchained

Oleg Kokorin
  • 2,288
  • 2
  • 16
  • 28
1

a brilliant answer here How to continue a docker which is exited from user kgs

docker start $(docker ps -a -q --filter "status=exited")
(or in this case just docker start $(docker ps -ql) 'cos you don't want to start all of them)

docker exec -it <container-id> /bin/bash

That second line is crucial. So exec is used in place of run, and not on an image but on a containerid. And you do it after the container has been started.

barlop
  • 12,887
  • 8
  • 80
  • 109
0

None of the answers address the point of this design choice. I think docker works this way to prevent these 2 errors:

  • Repeated restart
  • Partial error
Minh Nghĩa
  • 854
  • 1
  • 11
  • 28