Eventually I understand this and it works.
bash script:
#!/bin/bash
#$ -V
#$ -cwd
#$ -o $HOME/sge_jobs_output/$JOB_ID.out -j y
#$ -S /bin/bash
#$ -l mem_free=4G
c=$SGE_TASK_ID
cd /home/xxx/scratch/test/
FILENAME=`head -$c testlist|tail -1`
python testpython.py $FILENAME
python script:
#!/bin/python
import sys,os
path='/home/xxx/scratch/test/'
name1=sys.argv[1]
job_id=os.path.join(path+name1)
f=open(job_id,'r').readlines()
print f[1]
thx
Exported bash variables are actually environment variables. You get at them through the os.environ object with a dictionary-like interface. Note that there are two types of variables in Bash: those local to the current process, and those that are inherited by child processes. Your Python script is a child process, so you need to make sure that you export the variable you want the child process to access.
To answer your original question, you need to first export the variable and then access it from within the python script using os.environ.
##!/bin/bash
#$ -V
#$ -cwd
#$ -o $HOME/sge_jobs_output/$JOB_ID.out -j y
#$ -S /bin/bash
#$ -l mem_free=4G
c=$SGE_TASK_ID
cd /home/xxx/scratch/test/
export FILENAME=`head -$c testlist|tail -1`
chmod +X testpython.py
./testpython.py
#!/bin/python
import sys
import os
for arg in sys.argv:
print arg
f=open('/home/xxx/scratch/test/' + os.environ['FILENAME'],'r').readlines()
print f[1]
Alternatively, you may pass the variable as a command line argument, which is what your code is doing now. In that case, you must look in sys.argv, which is the list of arguments passed to your script. They appear in sys.argv in the same order you specified them when invoking the script. sys.argv[0] always contains the name of the program that's running. Subsequent entries contain other arguments. len(sys.argv) indicates the number of arguments the script received.
#!/bin/python
import sys
import os
if len(sys.argv) < 2:
print 'Usage: ' + sys.argv[0] + ' <filename>'
sys.exit(1)
print 'This is the name of the python script: ' + sys.argv[0]
print 'This is the 1st argument: ' + sys.argv[1]
f=open('/home/xxx/scratch/test/' + sys.argv[1],'r').readlines()
print f[1]
Take a look at parsing Python arguments. Your bash code would be fine, just need to edit your Python script to take the argument.
use this inside your script (EDITED per Aarons suggestion):
def main(args):
do_something(args[0])
if __name__ == "__main__":
import sys
main(sys.argv[1:])
