5

(I think[!]) I understand the principles behind pipenv (and the other venvs) and frequently use them. However, I've never really understood why pipenv needs both a Pipfile and a Pipfile.lock file.

This answer implies, and this tutorial says

Now, once you get your code and Pipfile.lock in your production environment, you should install the last successful environment recorded:

$ pipenv install --ignore-pipfile

but it doesn't explain why Pipfile.lock needs to be used. i.e. what does the .lock file contain that Pipfile doesn't, and why is Pipfile good enough to be shared with another developer:

Now let’s say another developer wants to make some additions to your code. In this situation, they would get the code, including the Pipfile, and use this command:

$ pipenv install --dev

but isn't good enough to be used to replicate your environment in production?

ChrisGPT was on strike
  • 127,765
  • 105
  • 273
  • 257
ChrisW
  • 4,970
  • 7
  • 55
  • 92

3 Answers3

19

The official Pipfile project has something to say about this:

The concrete requirements for a Python Application would come from Pipfile. This would include where the packages should be fetched from and their loose version constraints.

The details of the environment (all installed packages with pinned versions and other details) would be stored in Pipfile.lock, for reproducibility. This file will be automatically generated and should not be modified by the user.

In other words, Pipfile is for people and Pipfile.lock is for computers.

In your Pipfile you list the things you want, and you define them in a somewhat loose way like "Django version 2 or greater". But that's not good enough to deterministically reproduce an environment. Does that mean "Django 2.0.3" or "Django 2.1.0"?

Pipfile.lock specifies requirements exactly, and it also specifies dependencies exactly. For example, if you explicitly want foo and put that into your Pipfile, your Pipfile.lock will be generated locking that down to a specific version. If foo itself depends on bar, and bar depends on quux and florp, the Pipfile.lock file will lock bar, quux, and florp down too, so minor differences in dependencies don't break things.

ChrisGPT was on strike
  • 127,765
  • 105
  • 273
  • 257
  • 2
    That's what I suspected - so there's quite a lot of duplication between the 2 files? This still leaves some unanswered questions (in my understanding): why should another developer run ``pipenv install --dev`` rather than ``pipenv install --dev --ignore-pipfile``? – ChrisW Oct 23 '18 at 14:27
  • 7
    It depends on the goal. The `--ignore-pipfile` flag [tells `pipenv` to install locked dependencies from `Pipfile.lock`, ignoring the loose ones listed in `Pipfile`](https://pipenv.readthedocs.io/en/latest/basics/#pipenv-install). That's good for instances where you want the new environment to exactly match the one the project last used. But in development it is often fine, indeed encouraged, to apply minor updates, e.g. from Django 2.0.1 to 2.0.3. Then you can run tests and work using that new version. The upgrade will be locked for the next developer or deploy. – ChrisGPT was on strike Oct 23 '18 at 14:32
  • 2
    Also, "there's quite a lot of duplication between the 2 files"—yes, given that one is generated from the other (along with the versions of packages that are available at the time). – ChrisGPT was on strike Oct 23 '18 at 14:39
  • Ah, so if you don't specify a version and then install the environment and it doesn't work it would be at that stage you might specify a version in your ``Pipfile`` but you don't want to take that risk with production incase a new version has been released between your dev environment being setup and the production being released – ChrisW Oct 23 '18 at 16:15
  • 2
    @ChrisW, yes, mostly. If you let `pipenv` update dependencies by installing from `Pipfile` instead of `Pipfile.lock` and things break you could add a constraint. Better yet, you could fix your code so it works with the new libraries: frequently updating your libraries is usually a good idea. But your production environment should be as close as possible to the environment you've been using for development, testing, staging, etc. Installing dependencies from the lock file contributes to that, matching library versions and hashes for the whole dependency tree. – ChrisGPT was on strike Oct 23 '18 at 16:30
  • So when we write pipenv install --dev, then does it use pipfile or pipfile.lock? – variable Nov 10 '19 at 14:42
  • 2
    @variable, the `--dev` flag doesn't say anything about whether the lockfile should be used or not. [It tells Pipenv to install things in `dev-packages` as well as `packages`](https://pipenv.readthedocs.io/en/latest/cli/#cmdoption-pipenv-install-d). Unit test libraries, editor plugins, and other things you don't need in production at runtime are often defined as `dev-packages`. Whether Pipenv should install locked dependencies or not is controlled with other flags. If you just run `pipenv install --dev` your dependencies (and `Pipfile.lock`) will be updated. – ChrisGPT was on strike Nov 10 '19 at 15:04
1

As @Chris has said, Pipfile.lock is for computer while Pipfile is for human. If you take a look at Pipfile.lock file, you will find that every dependency even has sha256 codes!

That file is impossible for human to deal with, you can only deal with Pipfile. But Pipfile is not strict enough to reproduce a totally same environment. So that's why we also need a Pipfile.lock.

Sraw
  • 18,892
  • 11
  • 54
  • 87
  • So when we write pipenv install --dev, then does it use pipfile or pipfile.lock? – variable Nov 10 '19 at 14:43
  • @variable Both, it will update Pipfile first and then update the lock file correspondingly. – Sraw Nov 11 '19 at 06:29
  • The install command should use the pipfiles. Should they also update the pipfiles? I thought update happens only when we use 'pipenv install somePackage' – variable Nov 11 '19 at 07:04
  • @variable Oh, I am sorry. I misread your comment. Yes, `pipenv install --dev` will only use lock file. But I believe it will also warn you if your lock file doesn't match pipfile. – Sraw Nov 11 '19 at 07:06
0

This is a tool like npm.(maybe?)
The Pipfile is to identify the dependency of your project, you will get the dependency tree from the Pipfile.
But according to the different source, your will get different packages. So you can get actual local dependency from .lock file.
For example, in Pipfile, you can see something like:

matplotlib = "*"
numpy = "*"

But in .lock file, you will see the actual dependency like: 

"pytz": {
        "hashes": [
            "sha256:a061aa0a9e06881eb8b3b2b43f05b9439d6583c206d0a6c340ff72a7b6669053",
            "sha256:ffb9ef1de172603304d9d2819af6f5ece76f2e85ec10692a524dd876e72bf277"
        ],
        "version": "==2018.5"
    }

In short, Pipfile is to be more compatible, but .lock file is to get the actual dependency in local.

Rouzip
  • 101
  • 3