0

I'm working on a Javascript project, and as it so happens one of my dependencies pulls in puppeteer, which in turn downloads a whole copy of Chromium into my node_modules. My larger project is split into multiple Javascript packages, so I end up with multiple identical copies of Chromium among other stuff.

Is there a way to deduplicate these packages system wide? Note, npm dedupe seems to do something completely different to what I want.

I imagine there would be a module repository in my home directory which contains every package I need (in every version needed), and then in the local node_modules directories would contain only symlinks to the repository. This seems like an incredibly obvious optimisation, but I can't find any way to do it in npm. If not in npm, is it maybe possible in yarn?

As an added complication, this should also work on Windows (where symbolic link support has historically been not so good).

jdm
  • 9,470
  • 12
  • 58
  • 110
  • Does installing the modules globally (`npm i -g puppeteer`) not do what you desire? – Klaycon Jan 17 '20 at 22:57
  • No, I don't need it installed globally. I have multiple node.js projects that have different dependencies. I want `npm install` to provide the correct dependencies for each project, from `package.json`. But I want to get rid of the duplication of dependencies between different projects. It wastes space and more importantly, under Windows it makes things very slow. – jdm Jan 18 '20 at 00:38

1 Answers1

1

It seems the following command does what I want:

npm config set link -g

Then delete node_modules, and do npm install again. It should be much smaller now.

The documentation says:

If true, then local installs will link if there is a suitable globally installed package.

Note that this means that local installs can cause things to be installed into the global space at the same time. The link is only done if one of the two conditions are met:

  • The package is not already installed globally, or
  • the globally installed version is identical to the version that is being installed locally.

I am not sure if this has any negative side-effects - for example clobbering the global namespace with commands I don't want. For now, it seems to work fine.

Community
  • 1
  • 1
jdm
  • 9,470
  • 12
  • 58
  • 110
  • I followed these steps and don't see a reduction in a Vue3 applications node_modules size. The user is a allowed creating symlnks ([per group policy](https://stackoverflow.com/questions/894430/creating-hard-and-soft-links-using-powershell)). Tested with `new-item -Type SymbolicLink -Path .\node_modules_link\ -Value .\node_modules\` works without errors – theking2 Jun 22 '22 at 11:07