158

How do I execute a bash command from Ipython/Jupyter notebook passing the value of a python variable as an argument like in this example:

py_var="foo"
!grep py_var bar.txt

(obviously I want to grep for foo and not the literal string py_var)

denfromufa
  • 5,610
  • 13
  • 81
  • 138
Unni
  • 5,348
  • 6
  • 36
  • 55
  • Possible duplicate of [How to pass command line arguments to ipython](http://stackoverflow.com/questions/22631845/how-to-pass-command-line-arguments-to-ipython) – Amrith Krishna Feb 19 '16 at 07:47
  • 2
    @AmrithKrishna: Not really. See the edit. – Unni Feb 20 '16 at 04:27
  • Followup: I want to use keyword arguments such as `ffmpeg -i ...` but somehow the answer below doesn't work well with such a command. Do you know of an alternative solution? – axolotl Oct 01 '18 at 03:24
  • @Aalok: How about the edited answer? Can you please check now. If it doesn't work, then please post the entire command you are trying to run. – Unni May 06 '19 at 04:45
  • For those looking to pass the python variable to a bash cell (magic `%%bash`), see [this question](https://stackoverflow.com/q/19579546/10220019) – C. Binair Oct 13 '22 at 08:47

4 Answers4

222

General solution

As suggested by Catbuilts, use {..}:

dir_path = "/home/foo/bar"
!cp file1 {dir_path}

Its behaviour is more predictable than $... E.g. if you want to concatenate another string sub_dir to your path, $ wouldn’t give the expected result, while with {..} you can do:

!cp file1 {dir_path + sub_dir}

Initial (edited) answer

For simple cases, you can prefix your variable names with a $.

Example

Say you want to copy a file file1 to a path stored in a python variable named dir_pth:

dir_path = "/home/foo/bar"
!cp file1 $dir_path

from Ipython or Jupyter notebook

As mentioned above, note that it may lead to unexpected results depending on the content of your variables and the way you associate them, hence the general solution should be preferred.

Raw strings

For a related discussion on the use of raw strings (prefixed with r) to pass the variables, see Passing Ipython variables as string arguments to shell command

Skippy le Grand Gourou
  • 6,976
  • 4
  • 60
  • 76
Unni
  • 5,348
  • 6
  • 36
  • 55
  • 5
    While `!cp file1 $dir_path` and `!cp file1 {dir_path}` have similar result, I found that using `$` is a bit risky, because with `{}`, you can concatenate a path, something like `!cp file1 {folder + dir_path}`, whereas `!cp file1 $folder + $dir_path` deletes all your `folder` file. Be careful. – Chau Pham May 06 '19 at 03:46
  • 4
    If there are spaces in the variable, don't forget to add quotes!. For example if it is this`dir_path = "/home dir/foo/bar"`, use quotes like this: `!cp file1 "$dir_path"`. This adds quotes to the bash command. – Matt Bussing Oct 06 '19 at 02:11
  • Does not work among interpreters (e.g. if you do `%%bash echo $dir_path` instead of !echo $dir_path`) – Michele Piccolini Oct 17 '19 at 08:09
  • when using an environment variable in the same command, this doesn't work: `jupyter_var = 'jupyter', !echo $jupyter_var , os.environ['ENV_VAR'] = 'environment', !echo $jupyter_var, !echo $jupyter_var $ENV_VAR` gives `jupyter, jupyter, environment` – kingusiu Feb 10 '20 at 17:16
  • How is this accomplished when the shell expects a value inside a string? Example: `sp_details = !az ad app list --filter "DisplayName eq 'littleGuy'"`. If `sp_name = 'littleGuy'`, how to substitute `$sp_name` for `littleGuy` in the shell code? – SeaDude May 03 '20 at 07:12
  • @SeaDude: I am not sure if I follow. Neither am I too familiar with the command. You might want to post a separate question if you think that might help. – Unni May 03 '20 at 07:18
  • Done: https://stackoverflow.com/questions/61606054/passing-ipython-variables-as-string-arguments-to-shell-command – SeaDude May 05 '20 at 04:57
  • 1
    Here is the way to do this when IPython variable is a string inserted into a shell string: https://stackoverflow.com/questions/35497069/passing-ipython-variables-as-arguments-to-bash-commands – SeaDude May 09 '20 at 16:29
  • 5
    FYI for Colab users: the **`{variable}` notation doesn't work _the same way_ for Google Colab notebooks**. If your python `variable` contains spaces or anything that breaks cmd line commands, you still need to enclose it in quotes. So `!cp file1 "{dir_path}"`. Adding values of multiple variables does work - but quotes needed around it if it may have spaces: `!cp "{filename + filename}" "/content/drive/My Drive/"` – aneroid Dec 02 '20 at 14:53
  • Is this supposed to work in the pure ipython shell (CLI, not Jupyter notebook) as well? It didn't for me, but maybe that's because I had quoting as well in my case. – combinatorist Sep 10 '21 at 22:52
14

You cans use this syntax too:

path = "../_data/"
filename = "titanicdata.htm"
! less {path + filename}
sloik
  • 694
  • 5
  • 12
3

As @Catbuilts points out, $'s are problematic. To make it more explicit and not bury the key example, try the following:

afile='afile.txt'
!echo afile
!echo $PWD
!echo $PWD/{afile}
!echo {pwd+'/'+afile}

And you get:

afile.txt
/Users/user/Documents/adir
/Users/user/Documents/adir/{afile}
/Users/user/Documents/adir/afile.txt
paulperry
  • 826
  • 8
  • 16
  • 2
    There no simple solution to use both Python's and environment variables in one command. As a workaround, we can create a string variable as following: `command = f"echo $PWD/{afile}" !{command}` – Dmitry Balabka Feb 21 '21 at 21:46
0

Just an addition. In my case, and as shown in some of the examples in this question, my arguments were file names with spaces. Is that case I had to use a slightly different syntax: "$VAR". An example would be

touch "file with spaces.txt"
echo "this is a line" > "file with spaces.txt"
echo "this is another line" >> "file with spaces.txt"
echo "last but not least" >> "file with spaces.txt"
echo "the last line" >> "file with spaces.txt"
cat "file with spaces.txt"

# The variable with spaces such as a file or a path
ARGUMENT="file with spaces.txt"
echo $ARGUMENT

# The following might not work
cat $pwd$ARGUMENT

# But this should work
cat $pwd"$ARGUMENT"

I hope this helps. ;)

eliasmaxil
  • 530
  • 4
  • 13