Install dependencies globally and locally using package.json

Question

Using npm we can install the modules globally using -g option. How can we do this in the package.json file?

Suppose, these are my dependencies in package.json file

"dependencies": {
    "mongoose": "1.4.0",
    "node.io" : "0.3.3",
    "jquery"  : "1.5.1",
    "jsdom"   : "0.2.0",
    "cron"    : "0.1.2"
  }

When i run npm install, I want only node.io to be installed globally, the rest others should be installed locally. Is there an option for this?

Answer 1

New Note: You probably don't want or need to do this. What you probably want to do is just put those types of command dependencies for build/test etc. in the devDependencies section of your package.json. Anytime you use something from scripts in package.json your devDependencies commands (in node_modules/.bin) act as if they are in your path.

For example:

npm i --save-dev mocha # Install test runner locally
npm i --save-dev babel # Install current babel locally

Then in package.json:

// devDependencies has mocha and babel now

"scripts": {
  "test": "mocha",
  "build": "babel -d lib src",
  "prepublish": "babel -d lib src"
}

Then at your command prompt you can run:

npm run build # finds babel
npm test # finds mocha

npm publish # will run babel first

New NEW Note: For awhile now we have had npx, which allows you to run the devDependencies commands without needing to add them to your scripts section (if you want). For example:

npx webpack

But if you really want to install globally, you can add a preinstall in the scripts section of the package.json:

"scripts": {
  "preinstall": "npm i -g themodule"
}

So actually my npm install executes npm install again ... Which is weird but seems to work.

Note: you might have issues if you are using the most common setup for npm where global Node package installs required sudo. One option is to change your npm configuration so this isn't necessary:

npm config set prefix ~/npm, add $HOME/npm/bin to $PATH by appending export PATH=$HOME/npm/bin:$PATH to your ~/.bashrc.

Another, probably better option is to just use nvm to manage Node and you won't have that problem.

Answer 2

Due to the disadvantages described below, I would recommend following the accepted answer:

Use npm install --save-dev [package_name] then execute scripts with:

$ npm run lint
$ npm run build
$ npm test

My original but not recommended answer follows.

---

Instead of using a global install, you could add the package to your devDependencies (--save-dev) and then run the binary from anywhere inside your project:

"$(npm bin)/<executable_name>" <arguments>...

In your case:

"$(npm bin)"/node.io --help

This engineer provided an npm-exec alias as a shortcut. This engineer uses a shellscript called env.sh. But I prefer to use $(npm bin) directly, to avoid any extra file or setup.

Although it makes each call a little larger, it should just work, preventing:

• potential dependency conflicts with global packages (@nalply)
• the need for sudo
• the need to set up an npm prefix (although I recommend using one anyway)

Disadvantages:

• $(npm bin) won't work on Windows.
• Tools deeper in your dev tree will not appear in the npm bin folder. (Install npm-run or npm-which to find them.)

It seems a better solution is to place common tasks (such as building and minifying) in the "scripts" section of your package.json, as Jason demonstrates above.

Answer 3

This is a bit old but I ran into the requirement so here is the solution I came up with.

The Problem:

Our development team maintains many .NET web application products we are migrating to AngularJS/Bootstrap. VS2010 does not lend itself easily to custom build processes and my developers are routinely working on multiple releases of our products. Our VCS is Subversion (I know, I know. I'm trying to move to Git but my pesky marketing staff is so demanding) and a single VS solution will include several separate projects. I needed my staff to have a common method for initializing their development environment without having to install the same Node packages (gulp, bower, etc.) several times on the same machine.

TL;DR:

1. Need "npm install" to install the global Node/Bower development environment as well as all locally required packages for a .NET product.

2. Global packages should be installed only if not already installed.

3. Local links to global packages must be created automatically.

The Solution:

We already have a common development framework shared by all developers and all products so I created a NodeJS script to install the global packages when needed and create the local links. The script resides in "....\SharedFiles" relative to the product base folder:

/*******************************************************************************
* $Id: npm-setup.js 12785 2016-01-29 16:34:49Z sthames $
* ==============================================================================
* Parameters: 'links' - Create links in local environment, optional.
* 
* <p>NodeJS script to install common development environment packages in global
* environment. <c>packages</c> object contains list of packages to install.</p>
* 
* <p>Including 'links' creates links in local environment to global packages.</p>
* 
* <p><b>npm ls -g --json</b> command is run to provide the current list of 
* global packages for comparison to required packages. Packages are installed 
* only if not installed. If the package is installed but is not the required 
* package version, the existing package is removed and the required package is 
* installed.</p>.
*
* <p>When provided as a "preinstall" script in a "package.json" file, the "npm
* install" command calls this to verify global dependencies are installed.</p>
*******************************************************************************/
var exec = require('child_process').exec;
var fs   = require('fs');
var path = require('path');

/*---------------------------------------------------------------*/
/* List of packages to install and 'from' value to pass to 'npm  */
/* install'. Value must match the 'from' field in 'npm ls -json' */
/* so this script will recognize a package is already installed. */
/*---------------------------------------------------------------*/
var packages = 
  {
  "bower"                      :                      "bower@1.7.2", 
  "event-stream"               :               "event-stream@3.3.2",
  "gulp"                       :                       "gulp@3.9.0",
  "gulp-angular-templatecache" : "gulp-angular-templatecache@1.8.0",
  "gulp-clean"                 :                 "gulp-clean@0.3.1", 
  "gulp-concat"                :                "gulp-concat@2.6.0",
  "gulp-debug"                 :                 "gulp-debug@2.1.2",
  "gulp-filter"                :                "gulp-filter@3.0.1",
  "gulp-grep-contents"         :         "gulp-grep-contents@0.0.1",
  "gulp-if"                    :                    "gulp-if@2.0.0", 
  "gulp-inject"                :                "gulp-inject@3.0.0", 
  "gulp-minify-css"            :            "gulp-minify-css@1.2.3",
  "gulp-minify-html"           :           "gulp-minify-html@1.0.5",
  "gulp-minify-inline"         :         "gulp-minify-inline@0.1.1",
  "gulp-ng-annotate"           :           "gulp-ng-annotate@1.1.0",
  "gulp-processhtml"           :           "gulp-processhtml@1.1.0",
  "gulp-rev"                   :                   "gulp-rev@6.0.1",
  "gulp-rev-replace"           :           "gulp-rev-replace@0.4.3",
  "gulp-uglify"                :                "gulp-uglify@1.5.1",
  "gulp-useref"                :                "gulp-useref@3.0.4",
  "gulp-util"                  :                  "gulp-util@3.0.7",
  "lazypipe"                   :                   "lazypipe@1.0.1",
  "q"                          :                          "q@1.4.1",
  "through2"                   :                   "through2@2.0.0",

  /*---------------------------------------------------------------*/
  /* fork of 0.2.14 allows passing parameters to main-bower-files. */
  /*---------------------------------------------------------------*/
  "bower-main"                 : "git+https://github.com/Pyo25/bower-main.git" 
  }

/*******************************************************************************
* run */
/**
* Executes <c>cmd</c> in the shell and calls <c>cb</c> on success. Error aborts.
* 
* Note: Error code -4082 is EBUSY error which is sometimes thrown by npm for 
* reasons unknown. Possibly this is due to antivirus program scanning the file 
* but it sometimes happens in cases where an antivirus program does not explain 
* it. The error generally will not happen a second time so this method will call 
* itself to try the command again if the EBUSY error occurs.
* 
* @param  cmd  Command to execute.
* @param  cb   Method to call on success. Text returned from stdout is input.
*******************************************************************************/
var run = function(cmd, cb)
  {
  /*---------------------------------------------*/
  /* Increase the maxBuffer to 10MB for commands */
  /* with a lot of output. This is not necessary */
  /* with spawn but it has other issues.         */
  /*---------------------------------------------*/
  exec(cmd, { maxBuffer: 1000*1024 }, function(err, stdout)
    {
    if      (!err)                   cb(stdout);
    else if (err.code | 0 == -4082) run(cmd, cb);
    else throw err;
    });
  };

/*******************************************************************************
* runCommand */
/**
* Logs the command and calls <c>run</c>.
*******************************************************************************/
var runCommand = function(cmd, cb)
  {
  console.log(cmd);
  run(cmd, cb);
  }

/*******************************************************************************
* Main line
*******************************************************************************/
var doLinks  = (process.argv[2] || "").toLowerCase() == 'links';
var names    = Object.keys(packages);
var name;
var installed;
var links;

/*------------------------------------------*/
/* Get the list of installed packages for   */
/* version comparison and install packages. */
/*------------------------------------------*/
console.log('Configuring global Node environment...')
run('npm ls -g --json', function(stdout)
  {
  installed = JSON.parse(stdout).dependencies || {};
  doWhile();
  });

/*--------------------------------------------*/
/* Start of asynchronous package installation */
/* loop. Do until all packages installed.     */
/*--------------------------------------------*/
var doWhile = function()
  {
  if (name = names.shift())
    doWhile0();
  }

var doWhile0 = function()
  {
  /*----------------------------------------------*/
  /* Installed package specification comes from   */
  /* 'from' field of installed packages. Required */
  /* specification comes from the packages list.  */
  /*----------------------------------------------*/
  var current  = (installed[name] || {}).from;
  var required =   packages[name];

  /*---------------------------------------*/
  /* Install the package if not installed. */
  /*---------------------------------------*/
  if (!current)
    runCommand('npm install -g '+required, doWhile1);

  /*------------------------------------*/
  /* If the installed version does not  */
  /* match, uninstall and then install. */
  /*------------------------------------*/
  else if (current != required)
    {
    delete installed[name];
    runCommand('npm remove -g '+name, function() 
      {
      runCommand('npm remove '+name, doWhile0);
      });
    }

  /*------------------------------------*/
  /* Skip package if already installed. */
  /*------------------------------------*/
  else
    doWhile1();
  };

var doWhile1 = function()
  {
  /*-------------------------------------------------------*/
  /* Create link to global package from local environment. */
  /*-------------------------------------------------------*/
  if (doLinks && !fs.existsSync(path.join('node_modules', name)))
    runCommand('npm link '+name, doWhile);
  else
    doWhile();
  };

Now if I want to update a global tool for our developers, I update the "packages" object and check in the new script. My developers check it out and either run it with "node npm-setup.js" or by "npm install" from any of the products under development to update the global environment. The whole thing takes 5 minutes.

In addition, to configure the environment for the a new developer, they must first only install NodeJS and GIT for Windows, reboot their computer, check out the "Shared Files" folder and any products under development, and start working.

The "package.json" for the .NET product calls this script prior to install:

{ 
"name"                    : "Books",
"description"             : "Node (npm) configuration for Books Database Web Application Tools",
"version"                 : "2.1.1",
"private"                 : true,
"scripts":
  {
  "preinstall"            : "node ../../SharedFiles/npm-setup.js links",
  "postinstall"           : "bower install"
  },
"dependencies": {}
}

Notes

• Note the script reference requires forward slashes even in a Windows environment.

• "npm ls" will give "npm ERR! extraneous:" messages for all packages locally linked because they are not listed in the "package.json" "dependencies".

Edit 1/29/16

The updated npm-setup.js script above has been modified as follows:

• Package "version" in var packages is now the "package" value passed to npm install on the command line. This was changed to allow for installing packages from somewhere other than the registered repository.

• If the package is already installed but is not the one requested, the existing package is removed and the correct one installed.

• For reasons unknown, npm will periodically throw an EBUSY error (-4082) when performing an install or link. This error is trapped and the command re-executed. The error rarely happens a second time and seems to always clear up.

Answer 4

You could use a separate file, like npm_globals.txt, instead of package.json. This file would contain each module on a new line like this,

mongoose@1.4.0
node.io@0.3.3
jquery@1.5.1
jsdom@0.2.0
cron@0.1.2

Then in the command line run,

< npm_globals.txt xargs npm install -g

Check that they installed properly with,

npm list -g --depth=0

---

As for whether you should do this or not, I think it all depends on use case. For most projects, this isn't necessary; and having your project's package.json encapsulate these tools and dependencies together is much preferred.

But nowadays I find that I'm always installing create-react-app and other CLI's globally when I jump on a new machine. It's nice to have an easy way to install a global tool and its dependencies when versioning doesn't matter much.

And nowadays, I'm using npx, an npm package runner, instead of installing packages globally.

Answer 5

Build your own script to install global dependencies. It doesn't take much. package.json is quite expandable.

const { execSync } = require('child_process');
const fs = require('fs');

const package = JSON.parse(fs.readFileSync('package.json'));

let keys = Object.keys(package.dependencies);
let values = Object.values(package.dependencies);


for (let index = 0; index < keys.length; index++) {
    const key = keys[index];
    let value = values[index].replace("~", "").replace("^", "");

    console.log(`Installing: ${key}@${value} globally`,);
    execSync('npm i -g ' + `${key}@${value}`);
}

Using the above, you can even make it inline, below!

Look at preinstall below:

{
  "name": "Project Name",
  "version": "0.1.0",
  "description": "Project Description",
  "main": "app.js",
  "scripts": {
    "preinstall": "node -e \"const {execSync} = require('child_process'); JSON.parse(fs.readFileSync('package.json')).globalDependencies.forEach(globaldep => execSync('npm i -g ' + globaldep));\"",
    "build": "your transpile/compile script",
    "start": "node app.js",
    "test": "./node_modules/.bin/mocha --reporter spec",
    "patch-release": "npm version patch && npm publish && git add . && git commit -m \"auto-commit\" && git push --follow-tags"
  },
  "dependencies": [
  },
  "globalDependencies": [
    "cordova@8.1.2",
    "ionic",
    "potato"
  ],
  "author": "author",
  "license": "MIT",
  "devDependencies": {
    "chai": "^4.2.0",
    "mocha": "^5.2.0"
  },
  "bin": {
    "app": "app.js"
  }
}

The authors of node may not admit package.json is a project file. But it is.

Answer 6

All modules from package.json are installed to ./node_modules/

I couldn't find this explicitly stated but this is the package.json reference for NPM.

Answer 7

This probably may be door for problems in production. if project dependencies installed outside project folder, the code may break if anybody else delete or replace your packages or change folder permissions.

Having every thing in one folder is more durable and make system predictable and maintenance tasks easier.