0

Ok, this isn't the usual bash "command not found" question.

I have a generated environment file I build up with some bash scripting, then pass it into a docker-compose service definition. The file contains a couple of variables with single quotes and spaces in the values. Docker and my entrypoint script don't seem to mind, but I source the generated file in my script to create a couple of related files (update a properties file with some values) and when I source it I get "command not found" when those troublesome variables are read.

Here's a sample env file:

TEST1=Hi
TEST2=['top','bottom']
TEST3=Bye
TEST4=Hi Bye

When I try sourcing it, bash/shell tries to execute/eval "Bye" as if it was a call to set a variable before calling a command:

$ source ./test.env 
bash: Bye: command not found...

Is there way to import the variables without this evaluation?

Before you say it, I can't quote or escape these values like you might in normal bash. The generated file values work fine for docker and the node.js service running inside. I've found that trying different combination of quoting and escaping will work for bash but not node.js, etc...

For example if I double quote TEST4="Hi Bye" bash will be happy, but in my server.js when I process.env["TEST4"] the string contains the double quotes and they're impossible to remove, but when I use the variable the unwanted quotes are in there. Escaping \ also causes my problems because I have to actively read and replace them.

I just want bash sourcing to skip evaluation, or find another way to import the variables from file into a script. Any suggestions? Thanks.

dubmojo
  • 6,660
  • 8
  • 41
  • 68
  • 4
    `X=y z` is "run the z command with the environment variable X set to y" -- make sure to quote your strings – anthony sottile Aug 27 '20 at 07:02
  • 3
    Seems to be a duplicate of [Use key/value data from a file in a shell script](https://stackoverflow.com/questions/55941708/use-key-value-data-from-a-file-in-a-shell-script). The accepted answer says *Don't source files that aren't legal bash code.*, follow that advice, and parse `test.env` manually. Like `while read x; do declare "$x"; done – oguz ismail Aug 27 '20 at 07:03

1 Answers1

0

Well nobody seemed to have picked up on the node.js tag or the reason why I was trying to work with bash env variables (docker) with special characters that might break bash, but I appreciate the effort to comment and the casual up votes for "setting me straight" on simply not using illegal characters in sourced bash variables. I guess my point of being stuck in this situation was overlooked.

The answer is to use the dotenv module in Node.js in combination with a Docker entrypoint script that transfers any env variables (quoted, spaced or otherwise so bash is happy) into a .env file inside the container that dotenv can import:

if(fs.existsSync('/app/backend/.env')) {
    require('dotenv').config();

At which point you can simply read them from process.env. If I simply read from process.env without importing from a .env using dotenv, those same variables has embedded quotes that I can't remove and when I use the variable in something like a password or SQL query, the quotes show up.

My entrypoint script simply writes the env variables (in my case I only use vps_*) into a .env file that dotenv can import.

printenv | while read line; do
   key=${line%%=*}
   value=${line#*=}
   if echo "$key" | grep -q "^vps_"; then
      echo $key=\"$value\" >> /app/backend/.env
   fi
done

I hope this helps someone else who runs into this type of situation where you're stuck working with Node.js inside a container where shell env variables are being mis-read due to illegal bash characters, but the Docker .env file is generated and/or must be used with Bash. It's a misunderstood problem.

dubmojo
  • 6,660
  • 8
  • 41
  • 68
  • 1
    If you remove the quotes from your .env file it should still work with the `dotenv` package properly as well as the `--env-file` switch in docker. – Phil Dec 22 '21 at 16:23
  • Spaces in files do work with the `--env-file` switch but they cause a [`key-cannot-contain-a-space-error` message in docker compose v2](https://stackoverflow.com/a/70242918/1178971). I suppose it is ok to ignore this error if you are also loading the `.env` in via the switch. – forgetso Mar 14 '22 at 16:56