1

If you connect a Github project to a product like Cloudflare pages or Vercel, commits to the remote repo trigger new builds. These builds will run the appropriate install and build commands.

I haven't updated a site in months. But major changes have come to dependencies being used, and it's causing me so many headaches to try and go through one-by-one and address each and every issue that has surfaced.

I'm using pnpm, is there anyway I can have pnpm install look at the existing pnpm-lock.yaml so I can eventually build a project that is entirely the same as a previous build I had 6 months ago?

I just want to edit some text on my site and not have to make all these updates. I tried "freezing" the versions of all my dependencies and dev dependencies in package.json by removing instances of ^ to match what I see in my lock file, but that didn't work.

Alberto Salvia Novella
  • 950
  • 9
  • 16
user1087973
  • 800
  • 3
  • 12
  • 29
  • Does this problem only occur within a CI environment? If you delete `node_modules` locally and then `pnpm install` does it do what you want? – a2k42 Oct 09 '22 at 18:51
  • Ah, once I deleted the `node_modules` directory locally and then "freezed" the version of every dependency in my `package.json` according to `pnpm-lock.yaml` that seemed to work. – user1087973 Oct 12 '22 at 13:34
  • I was going to link to some previous answers about how npm lock files work, but I'm not convinced they were correct / up-to-date. If I can find time to do a proper investigation I will post an answer, but I take it for now you've sorted out the issue? – a2k42 Oct 13 '22 at 10:44
  • Well, I wrote a response but actually this [short answer](https://stackoverflow.com/a/53594050/1447419) sums it up pretty well. `npm ci` seems to be the same as `pnpm install --frozen-lockfile` which is on by default in CI environments. Couldn't see any difference by deleting the `node_modules/` folder, you just have to watch the "specifiers" and "dependencies" sections in `pnpm-lock.yaml` to see what's happening. – a2k42 Oct 13 '22 at 20:04

2 Answers2

7

Package Management with PNPM

See also why-does-npm-install-rewrite-package-lock-json

Semver

The semver specification explains how to use semantic versioning though you can probably skip to the npm docs.

As you probably know the numbers are in the form major.minor.patch. If you don't mind which patch release you have as long as it is the specified major and minor version you can use the ~ prefix. Similarly, to allow any minor version use ^.

Walkthrough

Inital Setup

pnpm init

pnpm add express

The package.json will contain (at time of writing):

"express": `"^4.18.2"`

A pnpm-lock.yaml is also created:

specifiers:
  express: ^4.18.2

dependencies:
  express: 4.18.2

express -> '.pnpm/express@4.18.2/node_modules/express'/

Using pnpm install

Giving it a first run without changing anything produces:

$ pnpm install
Lockfile is up to date, resolution step is skipped
Already up to date
Done in 653ms

Now if I change package.json to be exactly v4.16.0 we shall see an update to pnpm-lock.yaml

specifiers:
  express: 4.16.0

dependencies:
  express: 4.16.0

Adding the patch wildcard ~4.16.0 and running pnpm install again gives:

specifiers:
  express: ~4.16.0

dependencies:
  express: 4.16.0

Note that the install version did not change. If I delete the node_modules/ directory and reinstall, still no change.

Ok, now try updating the minor version in package.json to ~4.17.0.

specifiers:
  express: ~4.17.0

dependencies:
  express: 4.17.3

This time it did update the dependency and installed the latest patch version but did install the exact major and minor version. If you think about what the ~ means then this is expected.

The specifiers section in the lock file is just what we specify as the dependency in the package.json file. The dependencies section in the lock file should reflect the version that is installed, or will be installed.

If I delete the node_modules/ folder and pnpm install again then we still have 4.17.3.

Explanation

What confuses a lot of people about pnpm install/npm install is how the lock-file works with the semver specifier:

The installed version listed as a dependency in the lockfile must be compatible with the version specified in the package file.

  • If it is compatible, no changes will be made.

  • If it is incompatible, then the latest compatible version will be installed.

Perhaps because sometimes it seems to install the latest version, and not othertimes, the behaviour is not clear. To state this again, changes will only be made when there is an incompatibility between the packge version and lockfile version. The lockfile dependency never has the ~ or ^ wildcards because only one version is actually installed and that's what the lockfile is supposed to track.

Using --frozen-lockfile in a CI environment

The docs for pnpm install describe how the install will fail if the lockfile is out of sync or needs updating.

Changing the package.json back to ~4.16.0 and then doing the install:

$ pnpm install --frozen-lockfile
Lockfile is up to date, resolution step is skipped
 ERR_PNPM_OUTDATED_LOCKFILE  Cannot install with "frozen-lockfile" because pnpm-lock.yaml is not up to date with package.json

Note that in CI environments this setting is true by default. If you still need to run install in such cases, use "pnpm install --no-frozen-lockfile"

In fact, even if I specify the installed version exactly 4.17.3, because it differs to the specifier ~4.17.0, then it will err. The package.json and pnpm-lock.yaml are out of sync even though the version are compatible.

Finally I will make our package compatible with the latest version that was installed with the first pnpm add express command. To do this I use the minor version wildcard ^4.0.0 and unfreeze the lockfile with pnpm install --no-frozen-lockfile.

specifiers:
  express: ^4.0.0

dependencies:
  express: 4.17.3

While the specifier is updated to match the package file, the version is not chaged; it is compatible.

Running pnpm install --frozen-lockfile will work again, but not update the installed version.

Conclusion

In a normal environment the lockfile will determine the exact version installed unless it is not compatible with the package file, in which case it will install the lastest version specified by the package file.

In a CI environment the lockfile will not by default be updated and will need to be compatible with the package file for installs to occur.

If you want the latest version specified pnpm update will do the update to the lastest compatible version given in the package file.

Disclaimer

I've tested out everything here but it is complex and I have limited experience using pnpm in a real CI environment.

a2k42
  • 672
  • 6
  • 19
  • Thank you for the input. So what is the correct approach to maintain dependencies versions with pnpm? I've used yarn before with --frozen-lockfile in CI, pnpm seems to suggest the same by default yet it doesn't work. Is removing all the wildcards will be the proper choice here? Pnpm seems to lean towards it being the only choice, like in the OP's case. – Andrew Katsewich Apr 24 '23 at 14:58
-8

before deploying your project delete pnpm-lock.yaml

Dieumerci Md
  • 7
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Dec 04 '22 at 11:44
  • this seems to answer the opposite of what was asked – Adam Sparks Apr 30 '23 at 04:35