Revision Control and Dependency Resolution with NPM / Node / package.json

Question

We have not been committing node_modules folder(s) in our application to revision control. Our build processes and developer instructions include running "npm install" manually on an initial check out to install required node modules. Our package.json files detail specific dependency versions.

Recently, our automated builds broke because a down stream dependency broke due to a recent 3rd party commit which I did not think would be possible. Our package.json file is as follows:

{
          "name": "test-package",
   "description": "Test Package",
       "version": "1.0.0",
       "license": "UNLICENSED",
       "private": true,
    "repository": { "type": "svn", "url": "" },
  "dependencies": {
    "extend": "3.0.0",
    "windows-registry": "0.1.3"
  }
}

Specifically, our dependency on "windows-registry" version "0.1.3" broke because of a child dependency of that module ("ref" version "1.2.0"). The dependencies from "windows-registry" package.json file are as follows:

"dependencies": {
       "debug": "^2.2.0",
         "ffi": "^2.0.0",
         "ref": "^1.2.0",
  "ref-struct": "^1.0.2",
   "ref-union": "^1.0.0"
}

I would assume "windows-registry" would always reference version "1.2.0" of the "ref" package, but it was actually pulling in version "1.3.4" and then recently "1.3.5" which broke our builds. I verified in the package.json file for "ref" that it is not version "1.2.0". The package.json file for "ref" is huge and it has lots of values such as "ref@^1.2.0" under various keys within the file. Interesting parts of the package.json file are as follows:

{
  /* Lots of other stuff */
  "_spec": "ref@^1.2.0",
  "version": "1.3.4"
}

Why is NPM not loading the same consistent repeatable dependency graph? Should we be committing node_modules to our revision control?

kuzyn · Accepted Answer · 2017-09-07T16:03:52.947

See this SO answer:

In the simplest terms, the tilde matches the most recent minor version (the middle number). ~1.2.3 will match all 1.2.x versions but will miss 1.3.0.

The caret, on the other hand, is more relaxed. It will update you to the most recent major version (the first number). ^1.2.3 will match any 1.x.x release including 1.3.0, but will hold off on 2.0.0.

As far as your other questions: you should definitely not commit your node_modules folder. You should rather commit a package-lock.json file, which will freezes your dependencies as they are. The shrinkwrap command was typically used for this, but as of npm v5 the lock file is generated by default

I would also suggest looking into yarn, which is an npm compatible package manager that is better and faster at managing complex dependency trees

Finally, since the npm repository more or less enforce semver, it helps to be aware what each increment is supposed to mean in term of breaking vs non-breaking changes. In your case, the changes should have been backward compatible if the package author had followed semantic versioning:

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,

MINOR version when you add functionality in a backwards-compatible manner, and

PATCH version when you make backwards-compatible bug fixes.

The programmer in me wants to read the carret like a regex and it feels more specific to me - that's what I get for not reading the docs better. I guess with that said the only way to ensure your dependencies are consistent is to commit your node modules as well? That really stinks because it is large and contains compiled binaries in the bin folder etc. Are there any alternatives to this? — jriffel73, Sep 06 '17 at 20:11

Revision Control and Dependency Resolution with NPM / Node / package.json

1 Answers1