304

It seems that Pipfile / Pipfile.lock are intended to be replacements for requirements.txt, in the context of Python packaging. There isn't much documentation out there on how these actually work, however. I found an evolving description of pipfile on the PyPi section of the Python website here but it's pretty messy and doesn't explain the semantics of the different sections of the file.

Any pointers on how to understand these files?

BuZZ-dEE
  • 6,075
  • 12
  • 66
  • 96
Stephen
  • 8,508
  • 12
  • 56
  • 96
  • 3
    This is a direct analogue of `Gemfile` and `Gemfile.lock` from the Ruby world: The `.lock` file has specific versions for each dependency; the one without that extension has only the known-to-controlling-humans versions. That said, asking for an explanation of something that's still evolving and a long way off from being well-defined, much less standardized, is perhaps a bit premature. – Charles Duffy Sep 20 '17 at 19:29
  • (And similarly, the difference between `Pipfile` and `requirements.txt` is largely that the former tries to adopt features from the Ruby world, ie. being able to specify dependency sets for multiple environments and with conditions/options/etc. within a single file). – Charles Duffy Sep 20 '17 at 19:33
  • 3
    It seems like it's already been deployed in the Heroku "getting started with Python" repo (https://github.com/heroku/python-getting-started.git) so like it or not, seems like it's productionized. – Stephen Sep 20 '17 at 19:55
  • 2
    Gotcha. That said -- the docs look pretty solid to me. I don't know what I could write in an answer that wouldn't just be restating them. – Charles Duffy Sep 20 '17 at 19:56
  • What docs are you referring to? – Stephen Sep 20 '17 at 19:57
  • 1
    If you're referring to the link that I made in the OP then there are a number of things omitted, for example what does it actually mean for something to be in a section called source. – Stephen Sep 20 '17 at 20:22
  • I'd suggest [filing an issue](https://github.com/pypa/pipfile/issues) with the project, then, describing in detail the specific items which the documentation doesn't cover. That way you'll get directly to the folks most familiar with the codebase. – Charles Duffy Sep 20 '17 at 20:25

3 Answers3

344

The concept behind these files is simple and analogue to other already existing tools, if you have some familiarity with Ruby's Bundler or Node's Npm. Pipenv is both a package and virtual environment management tool that uses the Pipfile and Pipfile.lock files to achieve these goals.

Pipenv handles the virtual environment for you (no more activate and deactivate required). Below, some basics to get you started, see more at pipenv website.

Getting Started

Start using pipenv is easy, in your project folder type...

$ pipenv install

... and if it already has a requirements.txt file, it will generate a Pipfile file with the requirements and a virtual environment folder, otherwise, it will generate an empty Pipfile file. If you disliked or changed your mind about something that you have installed, just type...

$ pipenv uninstall <package>

... and you're good to go. To activate the virtual environment that pipenv already generated, go with...

$ pipenv shell

... and your virtual environment will be activated. To leave the environment...

$ exit

... and you will be back to your original terminal session.

Pipfile

The Pipfile file is intended to specify packages requirements for your Python application or library, both to development and execution. You can install a package by simply using...

$ pipenv install flask

... and it will be added as a dependency for deployment and execution or by using ...

$ pipenv install --dev pytest

... and it will be used as a dependency for development time. In both cases, if you need to be more specific about the package version, as stated in the documentation pipenv makes use of the same version specifiers used by pip. The file syntax is pretty straight forward, as follows.

[[source]] # Here goes your package sources (where you are downloading your packages from).
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages] # Here goes your package requirements for running the application and its versions (which packages you will use when running the application).
requests = "*"
flask = "*"
pandas = "*"

[dev-packages] # Here goes your package requirements for developing the application and its versions (which packages you will use when developing the application)
pylint = "*"
wheel = "*"

[requires] # Here goes your required Python version.
python_version = "3.6"

Pipfile.lock

The Pipfile.lock is intended to specify, based on the packages present in Pipfile, which specific version of those should be used, avoiding the risks of automatically upgrading packages that depend upon each other and breaking your project dependency tree.

You can lock your currently installed packages using...

$ pipenv lock

... and the tool will lookup your virtual environment folder to generate the lock file for you automatically, based on the currently installed versions. The file syntax is not as obvious as is for Pipfile , so for the sake of conciseness, it will not be displayed here.

dandev486
  • 4,143
  • 2
  • 14
  • 22
  • This is an interesting answer, but I think when I was asking I couldn't quite understand what is actually meant by `[[source]]`, like is that the source for the packages which are downloaded by pipenv or something else? I guess the rest of the two files is obvious enough. – Stephen Apr 17 '18 at 19:30
  • @Stephen, when using package managers such as dnf, npm and pip, the package manager have to know where from it will pull the packages that you require, that is the source. On python, by default, we can use the PyPI with pip, but we can also have a package server on premises (for instance a JFrog Artifactory install), so we can have multiple sources of packages from where we pull. By the way, PyPI just has updated it's interface, it is awesome! :) – dandev486 Apr 17 '18 at 23:34
  • Wait, why would pytest and coverage be in [packages] instead of [dev-packages]? Aren't these used by developers, rather than users? – Al Sweigart Nov 21 '18 at 17:46
  • @AlSweigart, it was just a random selection of packages from an already existing file in my computer, if you think it may be confusing for people reaching this answer, let me now so I can change them for clarity sake. In the original file, this setup is used so that my continuous integration pipeline can install packages required for running unit tests and code coverage. – dandev486 Nov 21 '18 at 18:47
  • 2
    I'll admit, I found it confusing because I'm working on a tutorial where I have to decide if the [package]/[dev-package] difference is important, or if I should just put everything in [package]. (Googling for this is what led me to this page.) But having both while putting dev stuff in [package] kind of threw me. I could see why requests should be in [package] though. – Al Sweigart Nov 21 '18 at 19:25
  • 1
    Changed, as requested, to avoid confusion on development packages, thanks for you comment, hope it is less confusing now :) – dandev486 Nov 21 '18 at 19:33
  • @XChikuX, as described [here](https://pipenv.readthedocs.io/en/latest/#pipenv-features), **the main commands are install, uninstall, and lock, which generates a Pipfile.lock**, but you can skip it by using the **--skip-lock** option if you don't want it for some reason ;) – dandev486 Dec 04 '18 at 22:44
  • Are we meant to also commit `Pipfile.lock` into source control? Especially when working with medium-sized and distributed teams? – Felipe Alvarez Dec 28 '18 at 03:55
  • 1
    I think it is a good idea, if you want to keep track of ones package versions along the source code evolution, since most of the packages will be version starred on `Pipfile`. If something breaks on a package upgrade, one should then be able to fix it by reviewing the package version history, and indeed, one of the creators of the requests library do recommend doing it [here](https://github.com/pypa/pipenv/issues/598). – dandev486 Dec 28 '18 at 09:12
  • Do you need a flag when using `pipenv install` to force pipenv to use `Pipfile.lock` to determine what packages to install, or does that just happen automatically? – Stephen May 06 '19 at 18:22
  • 1
    @Stephen, it is actually the opposite, as stated in [pipenv install documentation](https://docs.pipenv.org/en/latest/basics/#pipenv-install) you can specify the `--skip-lock` flag to force it to ignore the `Pipfile.lock` file rather than forcing it to use it ;) – dandev486 May 07 '19 at 19:57
  • what I want ask is that after 'pipenv lock' is done and I tried to add new things to the project after a long time and that time the packages that I used in that particular project got updated. When I hit something like 'pipenv install' the version of packages in both Pipfile and Pipfile.lock will be updates or it will stay as the same version that we locked even tho there will be new versions. –  Jan 04 '20 at 11:30
  • @ChinkySight, the `lock` file is intended to keep the exact same versions, so that your project keeps working, even after a while. So, when you `pipenv install`, the previously added package versions are preserved. – dandev486 Jan 04 '20 at 12:29
  • @ChinkySight, if you want to update, either a specific package or all packages though, you can run a `pipenv update `, where pkg is optional, as stated [here](https://pipenv-fork.readthedocs.io/en/latest/basics.html#example-pipenv-upgrade-workflow) – dandev486 Jan 04 '20 at 12:29
  • @danieldeveloper001 Thank you for your response I really appreciate that. But okay lets I have locked the lock file which contain openpyxl == 3.0.1 only. When I install the environment using 'pipenv install -ignore-Pipfile', it install the packages and set the V-environment it as 'Pipfile.lock' but real things is that when I perform 'pipenv install' on that it will update the openpyxl version to the latest which is 3.0.2 why is that ? –  Jan 04 '20 at 12:38
  • @ChinkySight, could you provide the steps that you've performed? I cannot reproduce the behaviour you are experiencing here. Below, the steps I've performed trying to reproduce the issue: (1) `pipenv install --python 3` ; (2) `pipenv shell` ; (3) `pipenv install openpyxl==3.0.1` ; (4) `pipenv install --ignore-pipfile` – dandev486 Jan 04 '20 at 12:47
  • 1
    @danieldeveloper001 my question was that, let assume that I locked the v-environment two months before where I had install the **'openpyxl'** only whose version was **3.0.1**. If I install the V-environment now via `pipenv install -ignore-Pipfile` then what will be the **openpyxl version,** 3.0.1 or 3.0.2 ? –  Jan 04 '20 at 13:39
  • @danieldeveloper001 **3.0.2** is the latest version :D –  Jan 04 '20 at 13:45
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/205375/discussion-between-chinky-sight-and-danieldeveloper001). –  Jan 05 '20 at 08:08
  • I am the OP. The answer as provided is OK, but one of the main things I was getting at when I posted a few years ago was my confusion about how `Pipfile.lock` actually locks anything, and I don't think it's convincingly addressed yet. My confusion was that I thought `Pipfile.lock` could lock my OWN environment at the versions that it contained, so I was confused when it got updated later. – Stephen Sep 17 '21 at 20:10
  • Now I think that that `Pipfile.lock` is only useful for the workflow where a new developer/system gets `Pipfile.lock` and runs `pipenv install`. In that case they should get the exact same packages. So in that sense it's a lock, but it never actually prevents any local package updates by developers who have it in their folder. Is that correct? – Stephen Sep 17 '21 at 20:11
  • @dandev486 If specified python version is 3.6 , then does it mean the latest 3.6 version e.g 3.6.15 or the version 3.6.0 ? – Aang Jul 25 '22 at 13:16
  • It has been a while since the last time I wrote Python, but from what I can infer from this [documentation](https://pipenv-fork.readthedocs.io/en/latest/basics.html#specifying-versions-of-python), there are both `python_version` and `python_full_version`, so I would assume that if not using the "full_version" option, it would use the closest version that matches the "version" option. Hence, using your example, 3.6 could mean any 3.6.x and using 3 would mean any 3.x.y. Can you test it and share the results? – dandev486 Jul 25 '22 at 16:18
  • "I thought Pipfile.lock could lock my OWN environment at the versions that it contained". Hello OP, from what I can recall, if a `Pipfile.lock` was created from an existing environment and a library foo was at version x.y.z, then the same number would appear in the lockfile. If changes were made to the `Pipfile` itself, the `Pipfile.lock` would be generated again, unless when the `--skip-lock` option was used (last documented option from this [link](https://pipenv-fork.readthedocs.io/en/latest/basics.html#pipenv-install)). – dandev486 Jul 25 '22 at 16:29
22

It is worth noting that as of now (2022), pipfile and pipfile.lock are only used by pipenv, a tool written by the same authors. There is currently no standard (i.e. PEP) that mentions those files.

Regarding pipenv there's other popular tools (like poetry) that try to solve a similar use case without the use of pipfiles.

On a related note, there's also the dawn of pyproject.toml supported by various PEPs like 660 and 621, which is slowly gaining support by setuptools, flit, hatch.

Update: Since Summer 2022 the python packaging tutorial replaced the setup.py and setup.cfg based tutorial completely with pyproject.toml.

Bastian Venthur
  • 12,515
  • 5
  • 44
  • 78
3

As explained above by @Charles Duffy, it is a direct analog of Gemfile and Gemfile.lock from the Ruby world. See reference below for more details.

Reference: https://medium.com/never-hop-on-the-bandwagon/gemfile-and-gemfile-lock-in-ruby-65adc918b856

Joshua Kan
  • 353
  • 2
  • 7