30

We have a private JFrog artifactory (name anonymised below) that npm is configured in a project root .npmrc -file:

registry=https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/

The resolved-field in the package-lock.json file shared via Git between developers is constantly changing between runs of "npm install" without any changes to package.json.

Some times a dl query parameter (pointing to the exactly same URL) gets added to the resolved URL:

- "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/@sailshq/lodash/-/lodash-3.10.3.tgz",
+ "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/@sailshq/lodash/-/lodash-3.10.3.tgz?dl=https://artifactory.jfrog.private.com/@sailshq/lodash/-/lodash-3.10.3.tgz",

Some times the query parameter points to npmjs.org registry:

 - "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/aproba/-/aproba-1.2.0.tgz",
 - "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/aproba/-/aproba-1.2.0.tgz?dl=https://registry.npmjs.org/aproba/-/aproba-1.2.0.tgz",

And some times the field points directly to npmjs.org repository:

- "resolved": "https://artifactory.jfrog.private.com:443/api/npm/npm-registry-virtual/acorn/-/acorn-3.3.0.tgz",
+ "resolved": "https://registry.npmjs.org/acorn/-/acorn-3.3.0.tgz",

Any of these changes may also go to the inverse direction.

This is really irritating, since it means we constantly have meaningless changes in package-lock.json, which causes merge conflicts and often prevents npm ci from executing correctly. npm cache clean --force does not seem to help. I know that npm install can resolve package-lock.json merge conflicts automatically, but that does not help with npm ci (since the whole point is to not run npm install in the CI environment). And, anyway, what is the benefit of seeing how the virtual npm registry resolves the packages internally (as I suspect is happening here)?

Is there some kind of configuration option to prevent JFrog Artifactory from making these kinds of changes to the resolved package URLs in a virtual npm registry? Or is it maybe a bug in npm?

Environment:

  • npm 6.11.3
  • JFrog Artifactory 6.10.6
Ville Heikkilä
  • 301
  • 3
  • 3
  • 1
    I don't see a way in which Artifactory can change the package-lock.json file. According to the npm documentation: package-lock.json is automatically generated for any operations where npm modifies either the node_modules tree, or package.json. I believe this behavior is related to the npm client – Dror Bereznitsky Sep 20 '19 at 15:16
  • seem like the same question as https://stackoverflow.com/questions/53127140/npm-lockfiles-shrinkwrap-get-random-dl-parameter-tacked-on-to-the-resolved-u – Dror Bereznitsky Sep 20 '19 at 15:24
  • 1
    Maybe these settings have something to do with the issue of changing resolve fields in package-lock.json? https://www.jfrog.com/confluence/display/RTF/Npm+Registry#npmRegistry-AdvancedConfiguration https://www.jfrog.com/confluence/display/RTF/Npm+Registry#npmRegistry-AutomaticallyRewritingExternalDependencies – Ville Heikkilä Sep 25 '19 at 08:09
  • It should effect the package-lock.json as the change is done inside Artifactory before the npm client is aware of the original URL. However it is worth checking it further – Dror Bereznitsky Sep 26 '19 at 19:17
  • 1
    @VilleHeikkilä did you ever figure out how to keep the registry consistent? – samando Jul 22 '20 at 18:57
  • @samando unfortunately, we never found the root cause. We've been using work-arounds (primarily running npm i when package-lock merge conflicts rise and using npm i instead of npm ci in the CI environment). – Ville Heikkilä Jul 24 '20 at 06:53
  • Does the same happens if you use `yarn`? – Михаил Нафталь Oct 16 '20 at 14:09
  • We switched to yarn and we haven't had the problem with yarn.lock. – Ville Heikkilä Oct 20 '20 at 10:46
  • Is the use of the public npm registry a surprise/bug? Or is that consistent with JFrog settings and expected behavior? – Trott Oct 21 '20 at 18:45

3 Answers3

4

I don't know why those alternate URLs are appearing or how to make them stop. But you can reduce (or maybe even eliminate!) the merge conflict pain for your developers by using npm-merge-driver. It was written by one of the devs who was employed on the npm cli team for years, and its sole purpose is to automate away package-lock.json merge conflicts.

Trott
  • 66,479
  • 23
  • 173
  • 212
  • 1
    It's a shame this got no-comment -1'ed because the suggested solution addresses exactly the problem the OP is experiencing. True, they're asking *why* those URLs appear and I don't know the answer to that (and it would probably take a current or former JFrog employee to say with any authority). But the package resolves the merge pain that results from this issue and not enough people know about it. – Trott Nov 05 '20 at 17:59
3

Our team has had success running npm ci first to ensure our locally pulled down and cached dependencies match the package-lock.json file.

Then, further npm installs should resolve as expected.

Michael Cortez
  • 31
  • 1
0

This sort of thing is normally caused by developers having slightly different versions of npm installed. Version 7 of npm just got released, so it is the perfect time to make sure the team all have exactly the same version installed.

If that doesn’t work try switching the team to yarn or pnpm.

David Bradshaw
  • 11,859
  • 3
  • 41
  • 70
  • 1
    This hasn't been my experience; same versions of NPM produced different resolved field on different machines. – Pavlo Oct 21 '20 at 08:42
  • We switched to `pnpm` and still have this issue! Just the churn is in the `pnpm-lock.yaml` file instead. – TranquilMarmot May 03 '21 at 21:29