46

My docker produces some temporary files.

Is there an encouraged strategy regarding those?

If I put those to /tmp, I'm not sure they'll get cleared.

Or should I expose the volume /tmp from the host machine?

sbrajchuk
  • 48
  • 6
Augustin Riedinger
  • 20,909
  • 29
  • 133
  • 206
  • Are your containers going to be really long running? They won't be cleaned as is because there's no cron in the container (unless you explicitly set it up). – johnharris85 Mar 15 '17 at 17:55
  • Yes, they are webservers, they are expected to be long-running. So I guess there is no best practice out-of-the-box for the moment, right? – Augustin Riedinger Mar 15 '17 at 18:28

1 Answers1

66

I am not aware of any encouraged way to manage temporary files with Docker as it will mostly depend on how you need to handle these temporary files with your application (should they be deleted on restart? Periodically?...)

You have several possibilities depending on your needs:

Use Docker tmpfs mount

You can mount a tmpfs volume which will persist data as long as the container is running (i.e. the data in the volume will be deleted when the container stops), for example:

docker run --mount type=tmpfs,destination=/myapp/tmpdir someimage

This may be useful if you (can) restart your containers regularly and the temporary data may be recreated on container restart. However if you need to be able to clean up temporary data while the container is running, this is not a good solution as you will need to stop the container to have your temporary data cleaned.

Edit: as per @alexander-azarov coment, the tmpfs volume size is unlimited by default with the risk of the container using up all the machine memory. Using tmpfs-size flag is recommended to mitigate that risk, such as docker run --mount type=tmpfs,destination=/app,tmpfs-size=4096

Writing into the container writable layer

The writable layer of the container is where all the data will be written in the container if no volume is mounted. It will persist on container restart, but will be deleted if the container is deleted.

This way the temporary data will be deleted only when the container is deleted. It may be a good solution for short-lived containers, but not for long-lived containers.

Mounting host machine /tmp in the container with a bind mount

For example:

docker run -v /tmp/myapp-tmp-dir:/myapp/tmpdir someimage

This will cause all data to be written in the host machine /tmp/myapp-tmp-dir directory, and result will depend on how the host machine manage /tmp (in most cases, data are cleared upon machine restart)

Create and mount a volume to manage data into

You can create a volume which will contain your data, for example:

docker run --mount source=myappvol,target=/myapp/tmpdir someimage

And manage the data in the volume: mount-it in another container and cleanup the data, deleting the volume, etc.

These are the most common solutions relying (almost) solely on Docker functionalities. Another possibility would be to handle temporary files directly from your software or app running in the container, but it's more an application-related issue than a Docker-related one.

Pierre B.
  • 11,612
  • 1
  • 37
  • 58
  • 3
    Thank you, personally I found your answer to be comprehensive. I have the only note that I think is worth mentioning: since a `tmpfs` mount resides in memory, there may be a risk to exhaust all the host's memory. – Alexander Azarov Mar 15 '19 at 14:32
  • Good point, this can be mitigated using `tmpfs-size` flag to limit the allocated size. I'll edit accordingly, thanks for the feedback – Pierre B. Mar 15 '19 at 17:07
  • 1
    How about mounting to `/tmp/myapp:/myapp/tmpdir` instead of `/tmp:/myapp/tmpdir`? The container will be not longer able to access the temp files of the host. – jrobichaud Nov 13 '19 at 19:43
  • is writing in host machine /tmp the most "efficient" (if you don't use tmpfs)? trying to resolve cannotinspect container errors in AWS (which seems to be about disk usage) – seanv507 Oct 10 '21 at 09:54
  • Efficiency in term of IO and speed may be better using tmpfs (from my own experience), but it may eat-up all your RAM. It depends on what efficiency you're looking for – Pierre B. Oct 11 '21 at 08:14
  • How to configure the tmpfs in the Dockerfile? – curiouscheese Oct 14 '22 at 08:04
  • Unfortunately you can't in Dockerfile, you can only declare volumes path there, specifying that it's a `tmpfs` must happen at runtime. You can see details on https://docs.docker.com/storage/volumes/ – Pierre B. Oct 14 '22 at 13:03
  • 1
    A note on using the container writable layer - it is easy to do and will work, but performance is generally worse than the other options. – afaulconbridge Oct 17 '22 at 10:14