384

I've taken some shared code and put it in an NPM module, one I don't want to upload to the central registry. The question is, how do I install it from other projects?

The obvious way is probably to set up my own NPM registry, but according to the documentation, that involves a lot of hassle.

Can I just install an NPM module that sits on the local filesystem, or perhaps even from git?

npm install --from-git git@server:project
Yves M.
  • 29,855
  • 23
  • 108
  • 144
futlib
  • 8,258
  • 13
  • 40
  • 55

16 Answers16

575

In your private npm modules add

"private": true 

to your package.json

Then to reference the private module in another module, use this in your package.json

{
    "name": "myapp",
    "dependencies": {
        "private-repo": "git+ssh://git@github.com:myaccount/myprivate.git#v1.0.0",
    }
}
heckj
  • 7,136
  • 3
  • 39
  • 50
250R
  • 35,945
  • 7
  • 33
  • 25
  • 64
    This is the *real* correct answer if you want your package.json to maintain the private repo dependency list, which is a Good Thing(tm) that you should be doing. –  Apr 24 '13 at 22:44
  • Thanks - this is perfect. One problem - it works if I don't include the # version at the end, but if I do, I get an error like this: "Error: Command failed: fatal: ambiguous argument 'v0.0.1': unknown revision or path not in the working tree." I can't find a solution to this. Do you know what's going on? – Jake Aug 14 '13 at 17:48
  • 7
    In the example it's referencing a specific tag, but if you don't have one then it'll default to master. (see http://git-scm.com/book/en/Git-Basics-Tagging) – 250R Aug 15 '13 at 00:17
  • 7
    I don't understand how this command line can download code from a private github repo if I am not supllying my credentials! So how can I pass my github credentials? – Renato Gama Dec 09 '13 at 19:57
  • You can setup the box with deploy keys (see https://help.github.com/articles/managing-deploy-keys) – 250R Dec 10 '13 at 23:49
  • can this be done without using ssh? I need to install on heroku not only on my local machine – SamAko Feb 13 '14 at 22:16
  • 35
    Note that the `"private": true` part isn't necessary, but it will help prevent your private repo from accidentally being published to the public npm registry. – evanrmurphy Sep 10 '14 at 18:14
  • @250R: What is the syntax if I always want to to download latest version of my custom package? – Navdeep Jul 29 '15 at 04:02
  • 4
    npm looks for several environment variables as well. From [the manual](https://docs.npmjs.com/cli/install) in the `npm install ` section, there are options like `GIT_ASKPASS` and `GIT_SSH`.An example usage to choose a key other than the default id_rsa: `GIT_SSH_COMMAND='ssh -i ~/.ssh/custom_ident' npm install git+ssh://git@github.com:npm/npm.git` – Jasmine Hegman Oct 19 '15 at 03:32
  • @250R : Does **npm** always pull from the master branch if a branch isn't specified in the dependency? What if I have another branch, say `develop` and I tag the commit there? I know **npm** will update a specific branch, if written like `.../myprivate.git#develop"`, but what if I want to pull a specific tag from that branch? Because just specifying the tag, like `.../myprivate.git#v1.0.1"` did not work. I got a `fatal: ambiguous argument 'v1.0.1': unknown revision or path not in the working tree.` – Milkncookiez Jan 14 '16 at 15:17
  • 1
    Is this possible for private bitbucket git repos? – chovy Aug 20 '16 at 00:41
  • This seems to hard code the location of the repository... is it possible to create a reference such that any user can check it out of their own repository without editing every reference in every package? – Michael Jan 12 '17 at 23:10
  • @Milkncookiez as already said by 250R the default branch is `master`, when no tag or branch is specified as url fragment. To me it works perfectly either with a branch name or a tag name; are you sure you correctly pushed the tag to *origin* via `git push --tags`? (and are you sure you are using the `v` prefix in your versioning tags?) – Kamafeather Jun 12 '19 at 13:49
  • I would like to know, what is the difference in using git repo as a private npm module versus npm private registry hosted ones. what are the key differences ? – Sandeep Vattapparambil Jan 20 '23 at 07:43
298
cd somedir
npm install .

or

npm install path/to/somedir

somedir must contain the package.json inside it.

It knows about git too:

npm install git://github.com/visionmedia/express.git
mihai
  • 37,072
  • 9
  • 60
  • 86
  • 4
    The path/to/somedir solution kind of works, but then it's kind of awful because all of the require statements then have to include that relative or absolute path. Please correct me if I'm doing something wrong... – Luke Bayes Oct 25 '12 at 03:40
  • 4
    @Luke yes, you're wrong. After `npm install` all the files are copied to your project directory. So the paths in the `require` statements will be relative only to your project directory. – mihai Oct 25 '12 at 07:52
  • 4
    I'm confused by the top part and the only reason I haven't tested this myself is that I'm still learning and don't have a private module to work on. Anyway, by changing your directory to where the module is and then calling `install` wouldn't that just install there and not for the project you want to use it for? – Adam Beck Jan 28 '13 at 04:48
  • @AdamBeck yes, that's right. That command is usefull for module developers who may want to do a quick test of the module to see if it works, or when you download a module and just want a quick Hello World. If you want to include it in your project you should definitely have a separate folder. – mihai Jan 28 '13 at 09:30
  • 13
    Side note: (a) when using git repos, you can specify a branch/commit/tag by adding a `#` to the end of the git url, eg `git://github.com/visionmedia/express.git#v0.0.1`; (b) To be safe add `"private": true` to the package.json of your private repos. This will make sure npm will never let you accidentally publish your secret sauce to the official npm registry. (according to http://debuggable.com/posts/private-npm-modules:4e68cc7d-1ac4-42d9-995a-343dcbdd56cb) – Rafael Xavier Mar 26 '13 at 19:32
  • 9
    FYI if you are serving your git up via http you'll need to `npm i git+http://all/the/things.git` even though `git clone http://all/the/things.git` works just fine – slf May 30 '13 at 19:09
  • But even if `path/to/somedir/` has a `package.json` file, it copies the directory itself into `{$PWD}/node_modules`. So now there is `{$PWD}/dependency/package.json`. and `${PWD}/node_modules/dependency` which is less than elegant. – Kevin Suttle May 22 '15 at 20:09
  • For `yarn`, the protocol is required. That is to say the prefix `git+ssh://` is required. – Sutra Aug 29 '17 at 03:59
  • @mihai Is it necessary to do a full npm purge of any existing package with the same name? Usually before I start experimenting with a fork of an NPM package, I already have it installed either globally or locally in my Node.js package. – Robert Oschler Jun 09 '18 at 13:48
  • You can actually add `"my-module-name": "file:../mymodule",` to your `dependencies` (assuming that _mymodule_ has a `package.json` file where it's indeed named `my-module-name`), run `npm install`, and then in your source files you can just reference your module by name, e.g. `import {util} from 'my-module-name';`. – Dániel Kis-Nagy Sep 13 '18 at 09:20
  • how to do something like `npm install git://home/michael/some.git`? that is, the repo is on the local file system, "git://" seems to insist on using the network, and without "git://" it fails because there's no package.json (due to it being a git repo) – Michael Aug 27 '19 at 16:38
68

Can I just install an NPM package that sits on the local filesystem, or perhaps even from git?

Yes you can! From the docs https://docs.npmjs.com/cli/install

A package is:

  • a) a folder containing a program described by a package.json file
  • b) a gzipped tarball containing (a)
  • c) a url that resolves to (b)
  • d) a <name>@<version> that is published on the registry with (c)
  • e) a <name>@<tag> that points to (d)
  • f) a <name> that has a "latest" tag satisfying (e)
  • g) a <git remote url> that resolves to (b)

Isn't npm brilliant?

Colonel Panic
  • 132,665
  • 89
  • 401
  • 465
  • 2
    Note that if you go for option b), it actually has to be a _gzipped tarball_, a mere zip archive won't do it. I.e. if you create your package with `tar -czf my-package.tar.gz dist` (assuming your `dist` folder also has a proper `package.json` file in it), then you can `npm install ../my-lib/my-package.tar.gz` from your other project. – Dániel Kis-Nagy Sep 17 '18 at 11:56
54

Update January 2016

In addition to other answers, there is sometimes the scenario where you wish to have private modules available in a team context.

Both Github and Bitbucket support the concept of generating a team API Key. This API key can be used as the password to perform API requests as this team.

In your private npm modules add

"private": true 

to your package.json

Then to reference the private module in another module, use this in your package.json

    {
        "name": "myapp",
        "dependencies": {
            "private-repo":
"git+https://myteamname:aQqtcplwFzlumj0mIDdRGCbsAq5d6Xg4@bitbucket.org/myprivate.git",
        }
    }

where team name = myteamname, and API Key = aQqtcplwFzlumj0mIDdRGCbsAq5d6Xg4

Here I reference a bitbucket repo, but it is almost identical using github too.

Finally, as an alternative, if you really don't mind paying $7 per month (as of writing) then you can now have private NPM modules out of the box.

arcseldon
  • 35,523
  • 17
  • 121
  • 125
  • Can you use this if you want to have a global module? – PI. Jan 13 '16 at 10:31
  • This does not work for me I am afraid. Both the remote and the repository are not found. Any ideas? – Thomas Bormans Feb 03 '16 at 15:50
  • @ThomasBormans - still having trouble? Please paste (scramble your teamname / api key) what you have as your line in dependencies section of your package.json - as per above instructions. I have found this works well for both github and bitbucket private repos. Which are you using? – arcseldon Feb 03 '16 at 17:50
  • @arcseldon _"name": "git+https://key:x-oauth-basic@bitbucket.org/user/repo.git"_ returns _EISDIR: illegal operation on a directory, read_. And _"name": "git+https://user:key@bitbucket.org/repo.git"_ returns several errors including these words _remote: Not Found, fatal: repository, Command failed: git clone_. Any ideas? – Thomas Bormans Feb 03 '16 at 17:59
  • Try to follow the "exact" format I have given in answer: "git+https://:@bitbucket.org/.git How did you generate the API key? Have you double checked it is correct in your bitbucket settings... sorry to ask the obvious, but i have near 100% confidence this should work. – arcseldon Feb 03 '16 at 18:09
  • @ThomasBormans - the team API key, which i refer to above, is available only with team accounts, and doesn't appear for an individual user account... you should see "API Key" under "Access Management" in the side bar of the settings page for a team account. However, for a user account "API Key" does not appear. It would be easy for you to create a team though (for which you are the only member) and then switch your needed repo to be owned by that pseudo-team and use its API Key (which you create at the link I just mentioned in settings). – arcseldon Feb 03 '16 at 18:16
  • @arcseldon The API key I am using is from a team account which I am member of :/ – Thomas Bormans Feb 03 '16 at 18:17
  • check you are referring to team name, and not your own user name in the values you are giving. otherwise, it 'should' just work... perhaps try generating a fresh API key... let me know how it goes. – arcseldon Feb 03 '16 at 19:31
  • @arcseldon No luck I am afraid. I checked the team name and regenerated the API key. – Thomas Bormans Feb 04 '16 at 07:36
  • sorry to hear that -.. running out of ideas - you were using "git+https:" as protocol, and you can git clone using that protocol? You are a member of the team to which you refer, and the repo is DEFINITELY also associated with that team etc.. etc. try to trouble shoot by asking yourself all these questions. Do think it is user error, as opposed to anything else at this point. Good luck with it. – arcseldon Feb 04 '16 at 18:22
  • @arcseldon I have figured it out! Apparently there were "old" fields in the package.json from an older Node version. I removed them and everything was fixed. – Thomas Bormans Feb 08 '16 at 20:40
  • @ThomasBormans - pleased to hear it! Note to self, when debugging something like this - mv package.json package.json.old && git init -y Then insert needed dependency and try again :) – arcseldon Feb 09 '16 at 00:11
30

FWIW: I had problems with all of these answers when dealing with a private organization repository.

The following worked for me:

npm install -S "git+https://username@github.com/orgname/repositoryname.git"

For example:

npm install -S "git+https://blesh@github.com/netflix/private-repository.git"

I'm not entirely sure why the other answers didn't work for me in this one case, because they're what I tried first before I hit Google and found this answer. And the other answers are what I've done in the past.

Hopefully this helps someone else.

Ben Lesh
  • 107,825
  • 47
  • 247
  • 232
  • 2
    Can you use a subfolder of the git repo? – Chris Oct 10 '14 at 18:28
  • Worked for me in 2019! But I had to ensure [git has credentials](https://stackoverflow.com/questions/5343068/is-there-a-way-to-skip-password-typing-when-using-https-on-github) to access that account. (E.g. test git clone twice with `https://`, and ensure the password is not needed on the second run. Then you are good to go!) – joeytwiddle Jan 28 '19 at 05:23
10

Structure your code in an accessible fashion like below. If this is possible for you.

  • NodeProjs\Apps\MainApp\package.json

  • NodeProjs\Modules\DataModule\package.json

Within MainApp @ NodProjs\Apps\MainApp\

npm install --S ../../Modules/DataModule

You may need to update package.json as:

 "dependencies": {
       "datamodule": "../../Modules/DataModule"
}

This worked for my situation.

Community
  • 1
  • 1
dynamiclynk
  • 2,275
  • 27
  • 31
9

I had this same problem, and after some searching around, I found Reggie (https://github.com/mbrevoort/node-reggie). It looks pretty solid. It allows for lightweight publishing of NPM modules to private servers. Not perfect (no authentication upon installation), and it's still really young, but I tested it locally, and it seems to do what it says it should do.

That is... (and this just from their docs)

npm install -g reggie
reggie-server -d ~/.reggie

then cd into your module directory and...

reggie -u http://<host:port> publish 
reggie -u http://127.0.0.1:8080 publish 

finally, you can install packages from reggie just by using that url either in a direct npm install command, or from within a package.json... like so

npm install http://<host:port>/package/<name>/<version>
npm install http://<host:port>/package/foo/1.0.0

or..

dependencies: {
    "foo": "http://<host:port>/package/foo/1.0.0"
}
bwest87
  • 1,223
  • 13
  • 12
5

Npm now provides unlimited private hosted modules for $7/user/month used like so

cd private-project
npm login

in your package json set "name": " @username/private-project"

npm publish

then to require your project:

cd ../new-project
npm install --save @username/private-project
actual_kangaroo
  • 5,971
  • 2
  • 31
  • 45
5

This was what I was looking for - get the latest from "private repo" :

GitHub :

$ npm install git+https://token:x-oauth-basic@github.com/username/my-new-project.git
$ npm install git+ssh://git@github.com/username/my-new-project.git

Bitbucket :

$ npm install git+https://username:password@bitbucket.org/username/my-new-project.git
$ npm install git+ssh://git@bitbucket.org/username/my-new-project.git
Bruno
  • 6,623
  • 5
  • 41
  • 47
Isaiah
  • 484
  • 5
  • 16
4

Starting with arcseldon's answer, I found that the team name was needed in the URL like so:

npm install --save "git+https://myteamname@aQqtcplwFzlumj0mIDdRGCbsAq5d6Xg4@bitbucket.org/myteamname/myprivate.git"

And note that the API key is only available for the team, not individual users.

Community
  • 1
  • 1
Kramer
  • 519
  • 1
  • 5
  • 13
4

I use the following with a private github repository:

npm install github:mygithubuser/myproject
Lars
  • 9,976
  • 4
  • 34
  • 40
3

Very simple -

npm config set registry https://path-to-your-registry/

It actually sets registry = "https://path-to-your-registry" this line to /Users/<ur-machine-user-name>/.npmrc

All the value you have set explicitly or have been set by default can be seen by - npm config list

sapy
  • 8,952
  • 7
  • 49
  • 60
3

You can use Verdaccio for this purpose which is a lightweight private npm proxy registry built in Node.js. Also it is free and open-source. By using Verdaccio it does not involve that much hassle as a plain private npm registry would.

You can find detailed information about how to install and run it on their website but here are the steps:

It requires node >=8.x.

    // Install it from npm globally
    npm install -g verdaccio

    // Simply run with the default configuration that will host the registry which you can reach at http://localhost:4873/
    verdaccio

    // Set the registry for your project and every package will be downloaded from your private registry
    npm set registry http://localhost:4873/

    // OR use the registry upon individual package install
    npm install --registry http://localhost:4873

It also has a docker so you can easily publish it to your publicly available docker and voila you have a private npm repository that can be distributed to others in a way as you configure it!

Dominik
  • 649
  • 7
  • 16
2

Config to install from public Github repository, even if machine is under firewall:

dependencies: {
   "foo": "https://github.com/package/foo/tarball/master"
}
Lex
  • 461
  • 3
  • 9
1

Obviously, setting up the private npm registry is the most scalable and long-term solution, although it's a bit of hassle in the beginning.

Also, you can install using the git+https/ssh as mentioned in the other answers. But if you have the private repo, and you're building the image in the cloud, let's say using google cloud build, you have to set up the GitHub ssh connection.

The simplest solution for one-off case like this can be solved using the following approach.

  • Clone and modify or create your own library from scratch.

  • Generate the archive file(package code along with its dependencies), using

    yarn install && yarn pack

    this will produce file like

    rich-markdown-editor-v11.13.117.tgz

  • move this file to libs folder and add this entry in dependencies object of package.json.

    "rich-markdown-editor": "file:libs/rich-markdown-editor-v11.13.117.tgz",
    
  • Now, install the package.

    yarn install
    
  • Make sure to add that file in your vcs and the installation process in docker image creation should work in cloud as well.

Note: if you frequently update the package and commit in your vcs, it will increase your repo size(while cloning with full history).

Krishna
  • 6,107
  • 2
  • 40
  • 43
1
  • Publish your module under an organization name using the standard "@my-org/my-module" (by default all organization modules are private).
  • From your npm profile create a read-only access token under "Access Tokens"
  • Next in your project directory root create a .npmrc file, and inside the file write the following:
//registry.npmjs.org/:_authToken=${Your_Access_Token}

Note: this also should work for others packaging services that follow the same standard.

Fareed Alnamrouti
  • 30,771
  • 4
  • 85
  • 76