Edit enviroment variables inside python for script bash

Question

my project, which uses mapreduce without hadoop, is composed of two files:

bash.sh and mapreduce.py.

I would like to use environment variables to communicate the information between bash.sh and mapreduce.py. Within bash.sh I use export myvariable and on mapreduce.py, I use os.environ ['myvariable'].

I would like to edit myvariable within mapreduce.py and print the result via bash.sh. I tried to execute this istruction: os.environ ['myvariable'] = 'hello', but on bash.sh 'myvariable' is empty. Do you have any suggestions?

As the answer by @Adii notes this is not possible. Environment variables are not global. They are private to each process. When one process starts another it can either pass it's current env vars to the new process or give it a custom set of vars. But that new process cannot modify the env vars of its parent process. More generally, no process can modify the env vars of another process. — Kurtis Rader, Aug 09 '19 at 19:03

Adiii · Answer 1 · 2019-08-09T11:40:39.107

You can't do it from python, but some clever bash tricks can do something similar. The basic reasoning is this: environment variables exist in a per-process memory space. When a new process is created with fork() it inherits its parent's environment variables. When you set an environment variable in your shell (e.g. bash) like this:

/why-cant-environmental-variables-set-in-python-persist

So you can only make available to Bash script if the bash script is called inside python process space. A simple example can be

bash script

#!/bin/bash
echo "var from python is $myvariable"

python script

import os
os.environ ['myvariable'] = 'hello'
print(os.environ['myvariable'])
# all environment varaibles will be availbe to bash script in this case
os.system('sh ./ab.sh')

This is the way that you can try. Otherwise, no way to set it and make it available to bash script.

Setting an environment variable sets it only for the current process and any child processes it launches. So using os.system will set it only for the shell that is running to execute the command you provided. When that command finishes, the shell goes away, and so does the environment variable. Setting it using os.putenv or os.environ has a similar effect; the environment variables are set for the Python process and any children of it.

ENV via python

You can also try vice versa as you mention in question

Here is sequence

First set in bash script
call python script from bash ( based on argument to avoid loop)
update ENV in python
call bash again from python, if you call outside it will vanish.

bash script

#!/bin/bash
export myvariable="hellobash"
echo "myvariable form bash $myvariable"
if [ ! -z $1 ]; then 
./py.py
else
echo "myvariable after updated from python $myvariable"
fi

Call bash script outside from python with the argument like myscript.sh bash, without argument in python myscript.sh

#!/usr/bin/python
import os
print("myvar form python",os.environ['myvariable'])
os.environ ['myvariable'] = 'hello'
print("myvar form python after update",os.environ['myvariable'])
os.system('sh ./ab.sh')

Edit enviroment variables inside python for script bash

1 Answers1