How do we call bash functions from python

Question

Let's say i have a bash function that i source into a shell.

# cat sample.sh 
function func()
{
    echo "Custom env : $CUSTOM_ENV"
}

Now i source this script in the bash shell: #source sample.sh

Then i define:

export CUSTOM_ENV="abc"

and then call func() from bash shell, it displays:

#func
Custom env : abc

Now if i am calling a python script from the same shell, i want to invoke the function func() from the python script. Anyway to achieve this ?

What i tried:

1. Tried os.system('func') - Doesn't work
2. Tried subprocess.check_output('func', shell=True, env=os.environ.copy()) - Doesn't work

Any guidance ?

Answer 1

You need to export the function (with the -f option):

$ function func()
> {
>     echo "Custom env : $CUSTOM_ENV"
> }
$ export -f func
$ export CUSTOM_ENV="abc"
$ python
Python 2.7.10 (default, Oct  6 2017, 22:29:07) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('func')
Custom env : abc
0

Note that exporting the function (and variables) exports copies of the function and variable to subprocesses of the shell they're exported from. They aren't available in parent (or sibling) processes. Also, changing them in a subprocess does not affect the original copy.

Also, exporting functions is a bash-only thing, so this only works if both the parent shell and the shell started from python are bash. On OSes where bash isn’t the default (e.g. recent releases of Ubuntu and Debian), you need to explicitly run bash, or it won’t work. This all makes it rather fragile, and as @triplee pointed pointed out, not a really good idea.

Answer 2

The problem you've got is that there are two separate bash processes involved:

• Outer bash -> knows about func, runs python
• Python, runs a new bash process when you call os.system or subprocess.check_output
• Inner bash -> created by Python, completely different from the outer bash.

Anything you've done in the outer bash is inaccessible to the inner bash. You need to feed the function to the inner bash.

Something like: subprocess.check_call("bash -c '. sample.sh && func'", shell=True)

That may not be exactly right, but you hopefully get the idea - you want the inner bash, created by Python, to learn about the function prior to trying to run func.

(Specifically in this code the third shell is probably spawning a fourth shell - it's easier that way but if you care about performance you may wish to tweak a bit.)

Answer 3

In your very specific example:

import subprocess
import os

def run_bash(cmd):
    subprocess.Popen(['/bin/bash', '-c', cmd])

#run_bash('ls -ltra')
#run_bash("date '+%A %W %Y %X'")

os.environ["CUSTOM_ENV"] = "MJ"
run_bash('./sample.sh')

Make sure sample.sh is made executable by chmod +x sample.sh

test_bash.py is:

import subprocess

def run_bash(cmd):
    subprocess.Popen(['/bin/bash', '-c', cmd])

run_bash('ls -ltra')
run_bash("date '+%A %W %Y %X'")

and

$ python test_bash.py 
Monday 50 2018 12:47:13 AM
total 668
-rw-r--r--. 1 jalal cs-grad   2590 Nov  4 16:46 data_loading.py
drwxr-xr-x. 8 jalal cs-grad    211 Nov  4 16:46 .git
drwxr-xr-x. 2 jalal cs-grad     85 Nov  4 22:44 .ipynb_checkpoints