how to use package.json to call a script that sets environment variables and have those variables accessible to other scripts and commands

Question

I have a cookiecutter Flask app. I am trying to modify my package.json so that the command npm start sets some environment variables with a Python script.

Right now my Python script appears to be behaving correctly, but when the script to start Flask is executed by package.json, the environment variables are not set.

I can start to articulate what is probably causing the problem. My Python script is probably running in a different process than Flask, and is setting the environment variables in its own process but Flask can't access them. I need to do some kind of analogue of sourcing my .bashrc. But I don't know what that analogue is. Or maybe the problem is something completely different!

Here's a chunk of my package.json:

"scripts": {
"build": "NODE_ENV=production webpack --progress --colors -p",
"start": "npm run set-trusted-vars-no-force && npm run start-servers",
"start-local": "npm run set-local-vars && npm start",
"start-servers": "concurrently -n \"WEBPACK,FLASK\" -c \"bgBlue.bold,bgMagenta.bold\" \"npm run webpack-dev-server\" \"npm run flask-server\"",
"webpack-dev-server": "NODE_ENV=debug webpack-dev-server --port 8081 --hot --inline $PUBLIC",
"flask-server": "flask run",
"lint": "eslint \"assets/js/*.js\"",
"set-trusted-vars": "python ../config/set_env_vars.py --reset T --env trusted --credentials ../config/credentials/secrets.yml --config ../config/config.yml",
"set-trusted-vars-no-force": "python ../config/set_dev_env_vars.py --env trusted --credentials ../config/credentials/secrets.yml --config ../config/config.yml",
"set-local-vars": "python ../config/set_dev_env_vars.py --reset T'",
"set-local-vars-no-force": "python ../config/set_dev_env_vars.py"

}

Here's a small relevant chunk of set_dev_env_vars.py:

def set_config(config_path, config_name):
    with open(config_path) as file:
        configs = yaml.safe_load(file)
        config = configs[config_name]
        for var in config:
            os.environ[var] = str(config[var])
            print(var, os.environ[var])

Here's what gets printed:

ENV local
FLASK_APP autoapp.py
FLASK_ENV development
SECRET_KEY not-so-secret
FLASK_RUN_PORT 8000
FLASK_RUN_HOST 0.0.0.0

Flask eventually complains:

[FLASK] Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.
[FLASK] npm ERR! code ELIFECYCLE
[FLASK] npm ERR! errno 2
[FLASK] npm ERR! cookiecutter_mbam@1.0.0 flask-server: `flask run`
[FLASK] npm ERR! Exit status 2

What's the right way to set environment variables so that Flask can find them?

p.s. I know in the absence of other context this might seem like an overengineered solution to setting environment variables, but I'm also dealing with more complicated cases that just aren't visible here.

Answer 1

My Python script is probably running in a different process than Flask, and is setting the environment variables in its own process but Flask can't access them.

Setting environment variables with os.environ['FOO']='bar' sets them for the duration of the process or its children (see this answer). Other threads discourage trying to make Python set them globally, among some hacky solutions.

Not only this, but if set_dev_env_vars.py did set the environment variables globally, this ties the frontend to the same machine as the Flask server. It is standard procedure to separate these two. I'm curious if you have a reason for starting the backend server from the frontend? Is this the behaviour of flask-cookiecutter or something you've implemented yourself (or are stuck with?)

Presumably config.yml contains the frontend config also. Perhaps you should separate the flask related configuration values from there and write them to a .env file and load them into your Flask application with python-dotenv. You could even make the frontend load its config with dot-env (javascript).

Having this separation would then allow you to deploy the frontend separately from the backend, by simply making sure the required .env file is available to the respective server.