Python bash magic: passing os.environ string argument with whitespace

Question

I have a function that requires a path string as argument that I want to call via bash magic from a jupyter notebook.

The path string contains a whitespace. When I pass the path string directly to the bash command, it works fine. However, when I set an os.environ variable, it gets truncated at the whitespace.

os.environ["DATA_PATH"] = "/content/drive/My Drive/data/"

%%bash
python -m my_function --filepath "/content/drive/My Drive/data/"
Recieved string: /content/drive/My Drive/data/

python -m my_function --filepath ${DATA_PATH} 
Recieved string: /content/drive/My

The strange thing is that when I echo both commands, I get the same results.

%%bash
echo python -m my_function --filepath "/content/drive/My Drive/data/" 
echo python -m my_function --filepath ${DATA_PATH} 

python -m my_function --filepath /content/drive/My Drive/data/
python -m my_function --filepath /content/drive/My Drive/data/

I have tried various things like escaping the whitespace, the quotes, etc. but I can't get it to work with the os.environ variable. Any help?

Answer 1

Double quoting the variable substitution should do the trick:

python -m my_function --filepath "${DATA_PATH}"

It is a question of how bash interpolates strings. There is a rather comprehensive answer here.