189

From what I understand about Docker, it's a tool used for virtual environments. In their lingo, its called "containerization". This is more or less what Python's virtualenv does. However, you can use virtualenv in Docker. So, is it a virtual environment inside a virtual environment? I'm confused as to how this would even work, so could someone please clarify?

nbro
  • 15,395
  • 32
  • 113
  • 196
danielschnoll
  • 3,045
  • 5
  • 23
  • 34
  • 63
    This is a good question, but will likely get closed as off-topic. virtualenv is not a real isolation, it's a poor man's isolation using path hacks and symlinks - you're still within your own operating system. Docker provides more isolation, but not as much as a full-on virtual machine. You could think of a container as a middle-ground between a virtualbox (heavy, expensive) and a virtualenv (light, cheap). Creating a virtualenv inside a container does not make much sense because the isolation is already provided by docker, there would be not much point in doing that. – wim Jun 21 '18 at 18:06
  • 3
    Possible duplicate of [what is the difference between vagrant, docker, virtualenv or just a virtual machine?](https://stackoverflow.com/questions/40177240/what-is-the-difference-between-vagrant-docker-virtualenv-or-just-a-virtual-mac) – phd Jun 21 '18 at 19:20

6 Answers6

217

A virtualenv only encapsulates Python dependencies. A Docker container encapsulates an entire OS.

With a Python virtualenv, you can easily switch between Python versions and dependencies, but you're stuck with your host OS.

With a Docker image, you can swap out the entire OS - install and run Python on Ubuntu, Debian, Alpine, even Windows Server Core.

There are Docker images out there with every combination of OS and Python versions you can think of, ready to pull down and use on any system with Docker installed.

sp0gg
  • 3,722
  • 1
  • 17
  • 20
  • 2
    Additionally there are [distroless images](https://github.com/GoogleContainerTools/distroless) for 'several popular programming languages' (including Python) from Google that 'contain only the programming language runtime' - from [ArchWiki/Docker](https://wiki.archlinux.org/index.php/Docker#Distroless) – muthuh Nov 22 '20 at 11:22
  • For virtual environments that encapsulate more than just Python dependencies (like Python itself), check out nixpkgs/nix-shell (also works on macOS). Docker on macOS for development/hot reloading of code using volumes is slow because on macOS, Docker runs inside a VM - and files/file system events need to be translated from the macOS file system to ext4 or similar. – Tobias Bergkvist Mar 01 '22 at 15:21
  • 1
    so if Docker swaps out entire OS, then isn't it become same as Virtual Machines ? – MaazKhan47 Feb 19 '23 at 19:37
  • @MaazKhan47 no, a virtual machine simulates the hardware (CPU, RAM) and redirects disk storage and network traffic from simulated memory to a file and simulated ports. Docker directly uses the underlying hardware for these resources. Storage and network ports are mapped into the container, RAM is sanctioned for the container. – dan Jun 27 '23 at 16:20
39

Python virtual environment will "containerize" only Python runtime i.e. python interpreter and python libraries whereas Docker isolates the whole system (the whole file-system, all user-space libraries, network interfaces) . Therefore Docker is much closer to a Virtual Machine than virtual environment.

jil
  • 2,601
  • 12
  • 14
  • 2
    Is there any benefit in creating a virtual environment inside a docker container considering that the container will only serve a flask web app. – thanos.a Apr 23 '20 at 22:04
19

Adding to the above: there is a case for combining docker and venv: some OSs ship with python installed to provide 'OS-near' apps, e.g., to my knowledge, apt on debian (and its derivatives). The python venv enables a developer to ship a python app which requires a different interpreter version without affecting the shipped-with-the-OS python. Now, since Docker 'isolates the whole OS' as stated above, the same applies to a Docker image. Hence, in my view, if a Docker image is required/desired, it is best practice to create a venv inside the Docker image for your python app.

Blindfreddy
  • 622
  • 6
  • 12
  • 6
    Would this slow down response time (two levels of virtualization)? – Andy Swift Feb 02 '19 at 13:08
  • 5
    Python virtual environment changes the python environment it does not virtualize the execution of the python interpreter. Docker container is not virtualized unless it is executed with a hypervisor (Docker Machine). – Morten Mar 20 '19 at 09:43
  • I still think that it will be tedious for me to mount docker inside an operating system, I usually do this, I program in a shell language all dependencies of the project external to pytohn and I execute them automatically, let's say in production via ssh – Alex Hurtado Sep 03 '20 at 15:30
  • _which requires a different interpreter version_ - This is false. The venv is the same version as the OS interpreter that created it – OneCricketeer Dec 06 '21 at 14:36
2

"a virtual environment, a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages"

A docker container provides a higher level of abstraction/isolation, it can has its own "process space, file system, network space, ipc space, etc."

Michael.Sun
  • 387
  • 5
  • 13
2

A virtual environment is the integration of a collection of dependencies that ensure one or more applications can seamless work together. It provides a set of runtime guarantees for the applications of interest. Virtual environments isolate the given set of dependencies from the system's applications allowing users and developers to have as many application contexts as desired.

In particular, Python virtual environments are designed to isolate a particular set of dependencies tied to a particular version of Python from the system Python. In this way, a user can have multiple system Python versions each of which can have a corresponding set of virtual environments each with independent dependencies. Because Python virtual environments only apply to Python, any non-Python application installed in the system will be seen by all Python virtual environments in the exact same way. In concrete terms, one can have Python3.7 to Python3.11 installed as system Pythons while at the same time having four Python3.10 virtual environments (venv1-venv4) each with different versions of the requests library. Applications within venv2 may only work with the particular version of requests in that virtual environment and no other.

On the other hand, Anaconda virtual environments extend the set of dependencies beyond Python to include virtually any application. This means that they can include system applications (non-Python) together with the complete set of underlying libraries completely independent of system applications. For example, an anaconda virtual environment on, say, Apple Silicon with the HDF5 library will have the complete build of HDF5 for Apple Silicon only within the virtual environment. Any application that is run outside this Anaconda virtual environment will be completely ignorant of the existence of HDF5.

Docker, or containers in general, are a completely different technology only available on Linux machines in which applications are virtually isolated from one another by namespaces. Each container provides a virtual process space only accessible by applications in that container, which are externally managed by the container runtime (e.g. Docker, Containerd, podman etc.). Therefore, it is possible to have containers with Anaconda virtual environments. The container runtime can also be ported to non-Linux OSes but a Linux kernel will still be needed to enable the underlying process namespaces. This is why Docker works on Windows and macOS even though both do not natively support process namespaces.

In summary, there are two main types of virtualisation:

  • application dependency virtualisation (Python and Anaconda virtual environments), which can be though of as 'static' virtualisation, and
  • process virtualisation through containers, which can be thought of as 'dynamic' virtualisation.

But, I could be wrong...

polarise
  • 2,303
  • 1
  • 19
  • 28
-1

First, containers is not the same as a virtual environment.

To better manage Python resources, it is recommended to create virtual environments and configure just packages for each version.

https://docs.python.org/3/tutorial/venv.html

When working with Docker containers you already perform process engineering encapsulation, you can review it on its official website

https://www.docker.com/resources/what-container/#

Eduardo
  • 1
  • 1
  • 1
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jan 25 '23 at 10:39