2

I am attempting to execute a BASH script which sets needed environmental variables from within a Jupyter notebook. I understand that the magic command %env can accomplish this, but the BASH script is needed in this instance. Neither the use of !source or %system accomplishes the goal of making the environmental variables persist within the Jupyter notebook. Can this be done?

user2153903
  • 61
  • 3
  • I'm not clear what you mean by *"persist"*. Do you mean they stay set across reboots? Or they stay set between different notebooks? Or they stay set from one cell to the next? What are you trying to do, please? – Mark Setchell Apr 02 '21 at 07:24

2 Answers2

1

You could use python to update os variables:

Cell

! echo "export test1=\"This is the test 1\"" > test.sh
! echo "export test2=\"This is the test 2\"" >> test.sh
! cat test.sh

Result

export test1="This is the test 1"
export test2="This is the test 2"

Cell (taken from set environment variable in python script)

import os

with open('test.sh') as f:
    os.environ.update(
        line.replace('export ', '', 1).strip().split('=', 1) for line in f
        if 'export' in line
)

! echo $test1 
! echo $test2

Result

"This is the test 1"
"This is the test 2"
J.M. Robles
  • 614
  • 5
  • 9
0

To permanently set a variable (eg. a key) you can set a Bash environment variable for your Jupyter notebooks by creating or editing a startup config file in the IPython startup directory.

cd ~/.ipython/profile_default/startup/
vim my_startup_file.py

The file will be run on Jupyter startup (see the README in the same directory). Here is what the startup .py file should contain:

  1 import os
  2 os.environ['AWS_ACCESS_KEY_ID']='insert_your_key_here'
  3 os.environ['AWS_SECRET_ACCESS_KEY']='another_key'

Now inside a Jupyter notebook you can call these environment variables, eg.

#Inside a Jupyter Notebook cell
import os
session = boto3.session.Session( 
    aws_access_key_id=os.getenv('AWS_ACCESS_KEY_ID'), 
    aws_secret_access_key=os.getenv('AWS_SECRET_ACCESS_KEY'),
    region_name='us-east-1'
)

You will need to restart your kernel for the changes to be created.

Proteinologist
  • 23
  • 5