python on linux can't find file inside function

Question

I'm writing a script on python 2.7.12 on linux with a bunch of functions that run external programs...

at some point, I have a function that runs a program, which generates an external file. The next step of the function is to read the external file and do some processing.

The function is

def RunProgram(input, options)
input_file=str(input)
options=str(options)
cmd=str('program + ' input_file + '--flag ' + options + ' --out temp.log &> /dev/null')
#print(cmd)
#print(cmd)
os.system(cmd)
with open('path_to_file/temp.log') as fp:
    for i, line in enumerate(fp):
        if i == 2:
            #print(line)
            solution=str(line) #stores 3rd line of log file
        elif i > 2:
            break
return solution

Somehow, although the external program runs and I see temp.log created from a bash shell, the function exits with the error

IOError: [Errno 2] No such file or directory: 'path_to_file/temp.log'

if in the function after os.system(cmd) I place

print(os.path.exists('path_to_file/temp.log'))
print(os.path.isfile('path_to_file/temp.log'))

I get 2 false, but if I run those commands after running the function and getting the error i get True for both and I see the file in the directory using ls.

Once I run the function again with temp.log already existing, the function works.

I have checked with os.getcwd() and the session is running in the correct dirrectory. I also checked I have read permissions on temp.log

..any ideas?

score 0 · Accepted Answer · answered Mar 13 '18 at 11:24

0

This is probably a race condition: When your code wants to read the output from the file, the program isn't finished yet and the output file doesn't exist yet. Try a loop with "sleep" waiting for the file you want to read to become available. Another possibility is that that the file permissions don't match.

answered Mar 13 '18 at 11:24

0x01

468
2
9

Thank you! Sleep does indeed remove the error, unfortunately this function is called in a loop.... I will do some sort of try statement then. – AndyG Mar 13 '18 at 11:33
1

Just leaving this here for future.. I ended up using the subprocess module instead of os.system . Format cmd as a list and then use `subprocess.check_call(cmd, stdout=DEVNULL, stderr=subprocess.STDOUT, close_fds=True)` as suggested in [https://docs.python.org/2/library/subprocess.html] and [https://stackoverflow.com/questions/11269575/how-to-hide-output-of-subprocess-in-python-2-7] – AndyG Mar 13 '18 at 13:51

python on linux can't find file inside function

1 Answers1