117

Because we don't want sensitive data in the project code, including the package.json file, using environment variables would be a logical choice in my opinion.

Example package.json:

  "dependencies": {
    "accounting": "~0.4.0",
    "async": "~1.4.2",
    "my-private-module":"git+https://${BB_USER}:${BB_PASS}@bitbucket.org/foo/bar.git"

Is this possible?

The question is not if this is wise or not good, just if it's possible.

KetZoomer
  • 2,701
  • 3
  • 15
  • 43
kaasdude
  • 1,336
  • 2
  • 8
  • 13

10 Answers10

67

In case you use .env file, let's use grep or eval to get a value environment variable from the .env file.

Updated start2 as @Paul suggested:

"scripts": {
    "start": "NODE_ENV=$(grep NODE_ENV .env | cut -d '=' -f2) some_script",
    "start2": "eval $(grep '^NODE_ENV' .env) && some_script"
}
Long Nguyen
  • 9,898
  • 5
  • 53
  • 52
  • This should be the top comment because it perfectly solves this. – Marko Rochevski Nov 09 '20 at 12:19
  • I've been using this but when you start using this in multiple places it gets very verbose. Is there a library that can make this shorter? – writofmandamus Dec 10 '20 at 20:03
  • this is awesome, thanks. it works in "scripts". unfortunately it doesn't work in "config" which would be really useful. any idea why? – bmacnaughton Dec 15 '20 at 13:48
  • nm - it's the shell, not npm – bmacnaughton Dec 15 '20 at 15:03
  • 1
    You can also shorten this to `eval $(grep '^NODE_ENV' .env) && some_script` – Paul Sep 16 '21 at 01:12
  • 14
    Two issues with this. 1: The question was about dependencies, not scripts. Scripts run in the shell so using an environment variable in a script is very different from using one in a regular package.json value. 2: (Maybe I'm missing something here, but) getting an environment variable's value via grep seems like a complicated way of just doing `$NODE_ENV`. Further, if `NODE_ENV` is already defined as an environment variable, then it doesn't need to be redefined. Seems like it just saves you from doing `source .env` (and you could use something like direnv to do it automatically for you). – Henry Woody Oct 22 '21 at 17:58
  • Is there any way to chain these like this: `eval $(grep -Ev '(^AWS_PROFILE|^AWS_REGION)' .env)` so that I don't have to do this: `eval $(grep '^AWS_PROFILE' .env) && eval $(grep '^AWS_REGION' .env)` – timhc22 Oct 20 '22 at 03:12
  • 1
    Shocked this has so many upvotes. Using `eval` to execute content from a file is a TERRIBLE idea. Hopefully you have time to find a new job before some malicious actor injects their `mysql -e "DROP DATABASE prod;"; NODE_ENV="123"` `.env` file into your company's build pipeline. – joshuakcockrell Feb 13 '23 at 20:55
  • what about in windows – Jahangir Hussain Aug 28 '23 at 10:33
22

I have similar but different requirement. For me, I want to use environment variables in the scripts.

Instead of using the environment variables directly in package.json, I do:

"some-script": "./scripts/some-script.sh",

And in some-script.sh:

#!/bin/sh

source .env

npm run some-other-script -- --prop=$SOME_ENV_VAR
Kostanos
  • 9,615
  • 4
  • 51
  • 65
techguy2000
  • 4,861
  • 6
  • 32
  • 48
10

Here's how I managed to work around package.json to achieve the same purpose. It uses a script that reads from a custom section of package.json for URL modules, interpolates environment variables in them, and installs them with npm install --no-save (the --no-save could be omitted, depending on the usecase).

As a bonus: it tries to read the env variable from .env.json, which can be gitignore'd, and very useful for development.

  1. Create a script that will read from a custom section of package.json

env-dependencies.js

const execSync = require('child_process').execSync
const pkg = require('./package.json')

if (!pkg.envDependencies) {
  return process.exit(0)
}

let env = Object.assign({}, process.env)

if (typeof pkg.envDependencies.localJSON === 'string') {
  try {
    Object.assign(env, require(pkg.envDependencies.localJSON))
  } catch (err) {
    console.log(`Could not read or parse pkg.envDependencies.localJSON. Processing with env only.`)
  }
}

if (typeof pkg.envDependencies.urls === 'undefined') {
  console.log(`pkg.envDependencies.urls not found or empty. Passing.`)
  process.exit(0)
}

if (
  !Array.isArray(pkg.envDependencies.urls) ||
  !(pkg.envDependencies.urls.every(url => typeof url === 'string'))
) {
  throw new Error(`pkg.envDependencies.urls should have a signature of String[]`)
}

const parsed = pkg.envDependencies.urls
  .map(url => url.replace(/\${([0-9a-zA-Z_]*)}/g, (_, varName) => {
    if (typeof env[varName] === 'string') {
      return env[varName]
    } else {
      throw new Error(`Could not read env variable ${varName} in url ${url}`)
    }
  }))
  .join(' ')

try {
  execSync('npm install --no-save ' + parsed, { stdio: [0, 1, 2] })
  process.exit(0)
} catch (err) {
  throw new Error('Could not install pkg.envDependencies. Are you sure the remote URLs all have a package.json?')
}
  1. Add a "postinstall": "node env-dependencies.js" to your package.json, that way it will be run on every npm install

  2. Add your private git repos to package.json using the URLs you want (note: they all must have a package.json at root!):

"envDependencies": {
  "localJSON": "./.env.json",
  "urls": [
    "git+https://${GITHUB_PERSONAL_ACCESS_TOKEN}@github.com/user/repo#semver:^2.0.0"
  ]
},

(the semver specifier #semver:^2.0.0 can be omitted, but refers to a git tag, which can be very useful, as it makes your git server a fully-fledge package manager)

  1. npm install
Christophe Marois
  • 6,471
  • 1
  • 30
  • 32
7

No, it's not possible. You should access the repo using git+ssh, and store a private key in ~/.ssh.

Your line then looks like:

"my-private-module":"git+ssh://git@bitbucket.org/foo/bar.git"

Which doesn't contain anything sensitive.

Steve Bennett
  • 114,604
  • 39
  • 168
  • 219
  • Additionally you can even use a different ssh key and ssh config for this purpose, not the usual id_rsa. – Zlatko Jan 07 '16 at 17:56
  • Yep, I'd recommend that. (I did mean that with "store *a* private key", but could be clearer.) – Steve Bennett Jan 07 '16 at 22:36
  • Thanks! Yeah, it's Heroku :-S. So should be a custom build pack I guess. Docker will be ultimate environment in the end I think. Need to do this! Regards! – kaasdude Jan 08 '16 at 17:52
  • This .npmrc is able to interpret environment variables though! But was not able to combine, not mend to use it for these purposes I think... – kaasdude Jan 08 '16 at 17:57
5

No it isn't possible as npm does not treat any string values as any kind of templates.

It may be better to just use git+ssh (if your provider supports it) with an ssh agent.

mscdex
  • 104,356
  • 15
  • 192
  • 153
  • anybody using docker for deployment shouldn't use this method – saurabh Dec 13 '19 at 02:52
  • @monothorn - what could be a better way to do that? Do you have any suggestion... I am still figuring out if we don't use ssh, what could be a better way. – Ashmah Jan 07 '20 at 10:12
  • @Ashmah HTTPS is the way to go but again you will have to restrict access for the token you have generated and make sure the repo is private. Other than these, you will have to understand if there are any more security concerns. – saurabh Jan 08 '20 at 13:03
3

I had the same need and my solution was based on @Long Nguyen's response. This way, I can only rely on what's defined on the .env file.

.env

...
SKIP_PREFLIGHT_CHECK=true
...

package.json

...
"scripts": {
  "test": "yarn cross-env $(grep SKIP_PREFLIGHT_CHECK ../../.env) react-app-rewired test --watchAll=false"
}
...
psergiocf
  • 1,493
  • 2
  • 20
  • 27
2

You can use environment values to inject in your package.json like this:

Any environment variables that start with npm_config_ will be interpreted as a configuration parameter. For example, putting npm_config_foo=bar in your environment will set the foo configuration parameter to bar. Any environment configurations that are not given a value will be given the value of true. Config values are case-insensitive, so NPM_CONFIG_FOO=bar will work the same.

https://docs.npmjs.com/misc/config#environment-variables

hakunin
  • 4,041
  • 6
  • 40
  • 57
  • 9
    This is for setting variables to be uses by npm scripts - similar to config section in package.json. These variables can't be read by package.json – kgiannakakis Jun 06 '18 at 06:58
2

You can install package https://www.npmjs.com/package/env-cmd and all your envs from .env file will be visible ie:

./.env:

ENV1=THANKS
ENV2=FOR ALL
ENV3=THE FISH

Package.json:

"scripts": {
    "test": "env-cmd pact-broker can-i-deploy --broker-token=${ENV1}"
  }

or another example from your question:

    "my-private-module":"env-cmd git+https://${BB_USER}:${BB_PASS}@bitbucket.org/foo/bar.git"
Darex1991
  • 855
  • 1
  • 10
  • 24
  • 1
    From the [env-cmd README](https://github.com/toddbluhm/env-cmd#-x-expands-vars-in-arguments): *"You must escape the `$` character with `\ ` or your terminal might try to auto expand it before passing it to `env-cmd`"*. He gives an example npm script: `env-cmd -x node index.js --arg=\\$VAR`. Also see https://github.com/toddbluhm/env-cmd-examples/blob/master/examples/env-file-env-expansion/package.json – icc97 Jun 15 '23 at 17:01
  • 1
    interesting, thanks :) I will check this later – Darex1991 Jun 20 '23 at 09:49
1

If you're running node inside a Docker container

Use Docker Compose to inject the env variable

app:
  environment:
    - NODE_ENV=staging

Run your package.json script from your Dockerfile

CMD [ "npm", "run", "start" ]

Use echo or printenv

 "scripts": {
    "start": "node -r dotenv/config app.js dotenv_config_path=/run/secrets/$(echo $NODE_ENV)"
    "start": "node -r dotenv/config app.js dotenv_config_path=/run/secrets/$(printenv NODE_ENV)"
}

Don't use this for sensitive env variables. It's a really good way to point to a Docker secrets file (like this example shows).

joshuakcockrell
  • 5,200
  • 2
  • 34
  • 47
0

For complicated environment variables, you can use https://stedolan.github.io/jq/ to access JSON file (env file at your case) JSON file could be something like

{
    "env" :
     {  
            "username" : "1345345",
            "Groups" :  [],
            "arraytest" : [
               {
                  "yes" : "1",
                  "no" : "0"
               }
             ]
     }
}

so the script could be something like this to access yes value

"scripts": {
    "yes": "jq [].arraytest[0].yes?"
}    
Amr
  • 433
  • 5
  • 12