Environment variables containing newlines in Node?

Question

I'm attempting to load an RSA private key into my nodejs application using environment variables, but the newlines seem to be being auto-escaped.

For the following, assume that the PRIVATE_KEY env var is set to the following (not my actual key):

PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\nMIICXAIBAAKBgQCqGKukO1De7zhZj6+H0qtjTkVxwTCpvKe4eCZ0FPqri0cb2JZfXJ/DgYSF6vUp\nwmJG8wVQZKjeGcjDOL5UlsuusFncCzWBQ7RKNUSesmQRMSGkVb1/3j+skZ6UtW+5u09lHNsj6tQ5\n1s1SPrCBkedbNf0Tp0GbMJDyR4e9T04ZZwIDAQABAoGAFijko56+qGyN8M0RVyaRAXz++xTqHBLh\n3tx4VgMtrQ+WEgCjhoTwo23KMBAuJGSYnRmoBZM3lMfTKevIkAidPExvYCdm5dYq3XToLkkLv5L2\npIIVOFMDG+KESnAFV7l2c+cnzRMW0+b6f8mR1CJzZuxVLL6Q02fvLi55/mbSYxECQQDeAw6fiIQX\nGukBI4eMZZt4nscy2o12KyYner3VpoeE+Np2q+Z3pvAMd/aNzQ/W9WaI+NRfcxUJrmfPwIGm63il\nAkEAxCL5HQb2bQr4ByorcMWm/hEP2MZzROV73yF41hPsRC9m66KrheO9HPTJuo3/9s5p+sqGxOlF\nL0NDt4SkosjgGwJAFklyR1uZ/wPJjj611cdBcztlPdqoxssQGnh85BzCj/u3WqBpE2vjvyyvyI5k\nX6zk7S0ljKtt2jny2+00VsBerQJBAJGC1Mg5Oydo5NwD6BiROrPxGo2bpTbu/fhrT8ebHkTz2epl\nU9VQQSQzY1oZMVX8i1m5WUTLPz2yLJIBQVdXqhMCQBGoiuSoSjafUhV7i1cEGpb88h5NBYZzWXGZ\n37sJ5QsW+sJyoNde3xH8vdXhzU7eT82D6X/scw9RZz+/6rCJ4p0=\n-----END RSA PRIVATE KEY-----"

If I call console.log directly with the preceding string, I get the following as output:

You can see that the newline chars are being respected. However, if I call console.log(process.env["PRIVATE_KEY"]), the output contains the \n literal instead of actual newlines:

I've been trying to find more information online about how node processes environment variables with no luck. How do I load this key via env vars while maintaining newline characters? If that's not possible, how can I restore them?

score 98 · Answer 1 · answered Apr 06 '16 at 01:19

98

Just replace \n before use the value:

var private_value = process.env.PRIVATE_KEY.replace(/\\n/g, '\n');
console.log(private_value);

will result correctly:

answered Apr 06 '16 at 01:19

Adriano Godoy

1,488
1
11
21

5

You're a genius! So simple perfect! – Thermech Nov 23 '16 at 01:20
3

Assumes that you have control of the application code however. – giorgiosironi Apr 26 '18 at 11:59
Brilliant! Simpy worked. In my case I used ("${System.env.SIGNING_KEY_ID}".replace(/\n/, '\n')) That nailed it. – Gal Rom Nov 28 '19 at 14:39
2

This worked, but I don't understand how. It's replacing `\n` with a `\n` ? – Nikhil Owalekar Dec 26 '19 at 04:59
1

@NikhilOwalekar it replaces \\n with \n. I had (thanks to @Adriano) the problem with my AWS lambda function and env var. – Kaymaz Mar 23 '21 at 21:34

JBallin · Answer 2 · 2019-09-13T07:30:01.353

20

dotenv supports newlines with double quotes:

double quoted values expand new lines

MULTILINE="new\nline" becomes
{MULTILINE: 'new
line'}

edited Sep 13 '19 at 07:30

answered Apr 29 '19 at 23:23

JBallin

8,481
4
46
51

1

This is definitely the accepted answer. Simple and documented. – Tien Do Sep 13 '19 at 04:12

Freewalker · Answer 3 · 2019-05-28T15:14:56.190

16

If you're using dotenv: We solved it this way, with newlines and JSON.parse (this allows any backslash escaped chars within the string, not just \n):

in .env:

MY_KEY='-----BEGIN CERTIFICATE-----\nabcde...'

in server.ts:

myKey = JSON.parse(`"${process.env.MY_KEY}"`), // convert special chars

See thread: https://github.com/motdotla/dotenv/issues/218

edited May 28 '19 at 15:14

answered Aug 25 '17 at 22:07

Freewalker

6,329
4
51
70

I used this code for some time but suddenly it stopped working and start showing Unexpected token in JSON at position 32 error which points towards the \ – spa900 May 28 '19 at 13:17
Getting `Unexpected token =` – nomadoda Jun 07 '19 at 13:20

mklement0 · Answer 4 · 2021-11-15T21:43:58.323

Node.js does correctly reflect environment variables that have embedded actual newlines, as the following bash snippet demonstrates:

$ PRIVATE_KEY=$'ab\ncde' node -p 'process.env["PRIVATE_KEY"].indexOf("\n")'
2  # 0-based index of the (first) actual newline char. in env. var. 'PRIVATE_KEY'

^{Note that $'...' is a special type of bash string in which escape sequences such as \n are expanded, so in the command above PRIVATE_KEY is indeed defined with 2 lines and passed as an environment variable to node (simply by prepending the variable assignment to the command to invoke, which is a standard feature in POSIX-like shells).}

A simple way to print the value of an environment variable verbatim is:

# With *verbatim* '\n'
$ X='line 1\nline 2' node -p 'process.env.X'
line1\nline 2

# With *actual newline*, thanks to $'...' interpolation.
$ X=$'line 1\nline 2' node -p 'process.env.X'
line 1
line 2

In fact, Node doesn't interpret the value of environment variables in any way (which is the right thing to do).

It must be that your PRIVATE_KEY variable doesn't contain actual newlines, but \n literals (a \ char. followed by char. n).

If the assignment command PRIVATE_KEY="..." in the question is a shell command, that would explain it: In POSIX-like shells such as bash, \n inside a "..." string is left as-is.
- By contrast, JavaScript "..." strings do interpolate escape sequences such as \n, which is why passing such a string directly to console.log() indeed outputs newlines; e.g., node -e 'console.log("ab\ncde")' indeed outputs 2 lines.
PRIVATE_KEY="ab\ncde" node -p 'process.env["PRIVATE_KEY"]' (literally) outputs ab\ncde, showing that \n was retained as a literal.

You have two options to fix your problem:

Preferably, define your environment variable with actual newlines to begin with - see below.
Alternatively, if you do not control how the environment variable is set, expand the \n literals to actual newlines in your Node.js (JavaScript) code: see Adriano Godoy's helpful answer.

To define PRIVATE_KEY with actual newlines in POSIX-like shells, use one of the following techniques:

bash, ksh, zsh: use an ANSI C-quoted string:

export PRIVATE_KEY=$'-----BEGIN RSA PRIVATE KEY-----\n....\n-----END RSA PRIVATE KEY-----'

others, such as dash (and from any shell script that uses sh):

export PRIVATE_KEY="$(printf %s '-----BEGIN RSA PRIVATE KEY-----\n....\n-----END RSA PRIVATE KEY-----')"

Alternatively, for better readability you can use a command substitution with a here-document:

export PRIVATE_KEY="$(cat <<'EOF'
-----BEGIN RSA PRIVATE KEY-----
...
...
-----END RSA PRIVATE KEY-----
EOF
)"

Finally, as Allen Luce's answer shows, POSIX-compatible shells allow even regular string literals to span multiple lines, though readability may suffer a bit:

export PRIVATE_KEY='-----BEGIN RSA PRIVATE KEY-----
...
...
-----END RSA PRIVATE KEY-----'

Thanks for being the only answer that really explains what's happening. — Seb D., Dec 04 '20 at 17:41

Allen Luce · Answer 5 · 2021-11-15T21:19:06.143

7

It may be easier to see what's actually going on by inspecting rather than printing:

% export X="hey\nman"
% node
> process.env['X']
'hey\\nman'
>

Node's REPL implicitly applies the util.inspect method which emits an escaped string. That shows the literal backslash and lower-case "n" that the environment variable contains. So what's being fed to console.log is more like this:

> console.log("hey\\nman")
hey\nman

One option (in most popular shells) is to embed newlines straight into a variable just by hitting enter within a quoted string:

% export X="hey
> man"
% node
> process.env['X']
'hey\nman'
> console.log(process.env['X'])
hey
man
undefined
>

This technique also works within script files, just use newlines within quotes. If you have to take the environmental variables as-is a common approach is use string replacement:

% export X="hey\nman\ndude\nwhat"
% node
> console.log(process.env['X'].replace(/\\n/g, '\n'))
hey
man
dude
what

edited Nov 15 '21 at 21:19

answered May 22 '15 at 18:46

Allen Luce

7,859
3
40
53

1

++, but the phrase "Node will ... escape them to avoid interpolation" is misleading and suggests a misconception. Node doesn't - and shouldn't - apply string interpolation to environment variables. The escaping you're seeing is an artifact of the Node REPL: when you print a string or variable just by submitting it, without `console.log()`, the REPL _at that point_ actively encodes the value as a JavaScript string for convenient reuse in JavaScript source code. Thus, if the REPL shows you `a\\nb`, it means that the string _literally_ contains `\n` (i.e., characters `\ ` and `n`). – mklement0 Apr 17 '16 at 02:15
The prompt char. (`%`) in your shell code suggest `csh`, but the use of `export` suggests a POSIX-like shell. In POSIX-like shells, `"hey\nman"` will _not_ create a newline between `hey` and `man`. Using `echo` to print the value can muddle the picture, because `echo` in some POSIX-like shells does interpret escape sequences. A shell-neutral way to print the value as-is is `printf %s "$X"`. To reiterate the previous point: whatever the value assigned by the shell is, Node.js will use _uninterpreted_ (even though the Node.js REPL may apply escaping for _display_ purposes). – mklement0 Apr 17 '16 at 02:34
Not sure how you thought I was saying `"hey\nman"` creates a newline, I'm demonstrating that it does not. The point about `echo` is a good one so I removed that. Also clarified to make it clear how Node escapes the string. – Allen Luce Nov 15 '21 at 21:24
That there was confusion around literal `\n` vs. actual newlines was evidenced by your since-removed `echo $X` example. I'm glad you removed it. A simple way to take diagnostic output out of the picture and echo the value verbatim: `X='hey\nman' node -p 'process.env.X'` vs. `X=$'hey\nman' node -p 'process.env.X'` – mklement0 Nov 15 '21 at 21:40

score 6 · Answer 6 · answered Feb 06 '20 at 18:40

As described in another topic you can encode yout multiline string in base64 and put it inside .env like

HTTPS_CA_CERTIFICATE=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURDVENDQWZHZ0F3SUJBZ0lVY29lWFp1R

And then decode it inside your app

var cert = new Buffer(process.env.HTTPS_CA_CERTIFICATE, 'base64').toString('ascii');

score 1 · Answer 7 · answered May 29 '19 at 15:22

1

I was using:

JSON.parse(`"${process.env.PUBPEM}"`)

which suddenly started to fail. So I changed it to:

JSON.parse(JSON.stringify(process.env.PUBPEM))

Which solved it for me.

answered May 29 '19 at 15:22

spa900

927
9
19

score 0 · Answer 8 · answered Dec 04 '19 at 12:26

For the benefit of anyone out there trying to use multi-line variable (i.e RSA private keys) in a dotenv file (should work for any configuration file though).

1) make each line of the private key end with a \n

  such that first line**\n**
  second line**\n**
  and the  third line and so on

2) now make the variable in single line in double quotes as follow KEY="such that first line**\nsecond line\nand the third line and so on\n**"

3) Ensure you add a leading and trailing \n character for the key as well KEY="\nsuch that first line**\nsecond line\nand the third line and so on\n**"

4) now in your code (i.e node, of course if you do not have access to code you have rely on the behaviour of the platform/app)

goodlyRepresentedKey = (process.env.KEY).replace(/\n/g, '\n');

(if biggly why not goodly)

Environment variables containing newlines in Node?

8 Answers8

Linked