2

The problem is as follows: I make 4-5 small websites per week, and use tooling (webpack, ejs, etc..). So every site I've been working on, I did npm install locally. It was OK with my old machine with HDD, but now I got a new laptop with SSD only (and have no chance to return to HDD). The point is, I'm afraid to kill my SSD drive too fast doing npm install 4 times a week. Is there any reasonable way to optimize this? Maybe can I install devDependencies globally or install it somewhere once and link to them, and to install locally only prod dependencies?

Here is my average package.json deps:

"dependencies": {
  "siema": "^1.5.1"
},
"devDependencies": {
  "autoprefixer": "^9.1.5",
  "copy-webpack-plugin": "^4.5.2",
  "ejs-compiled-loader": "^1.1.0",
  "group-css-media-queries-loader": "^2.0.2",
  "postcss-loader": "^3.0.0",
  "babel-core": "6.26.3",
  "babel-loader": "7.1.5",
  "babel-polyfill": "^6.0.16",
  "babel-preset-env": "1.7.0",
  "babel-preset-stage-0": "^6.0.15",
  "clean-webpack-plugin": "0.1.19",
  "cross-env": "5.2.0",
  "css-loader": "1.0.0",
  "file-loader": "1.1.11",
  "html-webpack-plugin": "3.2.0",
  "jimp": "^0.6.0",
  "mini-css-extract-plugin": "^0.4.3",
  "node-sass": "4.9.2",
  "optimize-css-assets-webpack-plugin": "^5.0.1",
  "puppeteer": "^1.11.0",
  "raw-loader": "^1.0.0",
  "sass-loader": "7.0.3",
  "style-loader": "0.21.0",
  "uglifyjs-webpack-plugin": "^2.0.1",
  "url-loader": "1.0.1",
  "webpack": "4.16.3",
  "webpack-cli": "3.1.0",
  "webpack-dev-server": "^3.1.10",
  "webpack-merge": "4.1.3"
}

Thanks in advance.

UPDATE: Probably solved.

Referring to this thread, I've installed my devdeps at separate folder close to root, and at project's package.json I've linked devdeps as follows:

"devDependencies": {
  "autoprefixer": "file:c:/_npmg/node_modules/autoprefixer",
  "copy-webpack-plugin": "file:c:/_npmg/node_modules/copy-webpack-plugin",
  "ejs-compiled-loader": "file:c:/_npmg/node_modules/ejs-compiled-loader",
  /* ... */
}

Results are like this:

node_modules: size and structure

...and everything is working just fine.

Now I wonder if this is the most proper solution for my case (considering that it is only local dev need, I don't use docker or something) or is there more optimal way.

Alex Naidovich
  • 188
  • 4
  • 15
  • What are you delivering? You're uploading/delivering the whole source of the websites? Or only the `dist` folder? Your delivering this to clients or just uploading the `dist` to a remote server? Do your clients/other people using your sites to install them anew, or you may skip the dev dependencies in the `package.json` of your websites? – Sergeon Mar 04 '19 at 14:07
  • @Sergeon I deliver `dist` folder only, upload bundles to ftp static hosting. I do not strive to deliver source (if I got you right). – Alex Naidovich Mar 04 '19 at 21:13
  • 2
    You can try the `pnpm` https://pnpm.js.org package installer. it keeps a copy of every version of any package you have installed on your machine in a global store and links any future installs to the store instead of installing again. – Sachin Gupta Mar 05 '19 at 09:47

2 Answers2

2

Update:

My original solution below is hacky, and has some cons. On the other hand, pnpm, as suggested by @sachin Gupta, seems the straightforward solution for your problem. I didn't try it yet, but if the documentation is not lying, is really as easy as following the installation guide and then just use the command $ pnpm instead of $ npm to install your project's dependencies.

Storing common dependencies in a root folder

A way to solve your problem is to have all your project folders to be subfolders of another -let's call it root-, which in turn also has a package.json and a node_modules. This root project has all the common dev dependencies that your projects share: webpack, babel, node-sass etc.

Folder structure example

. root/
+-- node_modules/
+-- package.json
|   +-- web-project-1/
|     +-- package.json
|     +-- src/
|       +-- main.js
|       +-- index.html
|     +-- dist/
|       +-- index.html
|   +-- web-project-1/
|     +-- package.json
|     +-- src/
|       +-- main.js
|       +-- index.html
|     +-- dist/
|       +-- index.html

This is just an example. The only important part is that the dev dependencies are in a node_modules folder of a parent of your projects.

The good parts

  1. Any import or require in the children folders will use the root's node modules. So your application code will consume parent's node_modules without the need for further configuration. that's the default behavior of node.js when you perform a require in any folder tree: if it does not find a node_modules in the working directory, it always looks for node_modules in parent directories until the root folder.

  2. Any npm script in the children that use node_modules binaries will use them too. That's neat. So if in any of the children folders you have a project with webpack as an npm script defined in the package.json, you can call it from command line: npm run webpack without the need to have node_modules installed in the child folder.

So, in the example, if root's node_modules has a tslint dependency, you can setup web-project-1/package.json scripts like:

"scripts": {
  "lint": "tslint --project . --fix"
},

And when you run npm run lint in the terminal, the root's node_modules tslint binaries will be used to perform that command.

The same happens for every other build-step dev dependency, like webpack, gulp, node-sass or the likes.

The bad parts

The bad part is that, if you npm install any of the children from their package.json, the dependencies listed in that package.json will be installed in the children's too, even if they are already present on the parent. This make this approach pretty nasty if you want to declare the children's dev dependencies in their package.json: you should then manually remove the duplicated deps in the children every time you perform an npm install.

However, since you're only delivering the dist folder, you don't really need to list the dev dependencies of every project in its package.json. If you ever need to work with another dev, you can pass the root's package-json to them so they get the dev dependencies right.

The advantage of this approach over your's is that you don't need to manually setup the symlinks in every package-json of a new project.

Key points

  • The parent must have all the dev dependencies of the projects.
  • You can't have the dev dependencies listed on the children package.json, unless you're willing to manually delete them from node_modules after every npm install, which is too much of a hassle I believe.
  • If you ever need another version of a dependency in one specific project, you actually can list it in the package.json: it will get installed in the children's node_modules.
Sergeon
  • 6,638
  • 2
  • 23
  • 43
  • Adding to the "Bad parts", intellisense of IDEs may get affected and some features, like autocomplete, may not work properly. – Sachin Gupta Mar 05 '19 at 09:51
  • mmm, actually, as far as I know, if node can resolve a dependency, the IDE should too. I tried a small setup for this with a toy node/typescript project and it worked fine. However, more complex things, like trying to import a vue component defined in a sfc .vue file on the root, or importing any other thing that involve a babel plugin, may fail. I am really not sure. – Sergeon Mar 05 '19 at 12:30
  • I actually tried using this method when working with angular. I would skip the package installation in cli and placed node_modules at the root of the drive. The intellisense of vscode wasn't working well while importing the modules. – Sachin Gupta Mar 05 '19 at 13:09
2

I was in a similar position, as I am (maybe too much) very conscious about my SSD write cycle limits and was worried that all the npm install's would cut the lifetime short.

PNPM to the rescue!

I was reluctant at first, as I usually am before trying something new, but once I tried it, I'm never going back to pure npm (unless of course they make it work like pnpm by default).

The documentation on their website doesn't lie, the installation is as simple as running one command:

npx pnpm add -g pnpm

And then you can use it like you would normally use npm. The node_modules folder is so much cleaner, because it doesn't contain every single dependency of a dependency, but just folder aliases to the actual dependencies of your project.

pnpm install

Is so much faster, especially if you already have a bunch of dependencies installed, because then it's just about linking them to the project, without having to download them all again.