FileNotFound Error while runing subprocess.run in Python

Question

FileNotFound Error arose while I was executing subprocess.run() through the code below in Python:

from subprocess import run, DEVNULL, PIPE

def Befehl_Schreiber():
    flag_op = 'DEVNULL'
    flag_err = 'PIPE'
    befehl = 'usearch11 -threads 8 -fastq_filter ' + '.\Rohrdaten\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001.fastq' + ' -fastq_maxee 1 -fastq_minlen 120 -fastq_maxns 0 -fastaout ' + '.\\1_QC\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001--1.QC.fasta'
    return "'" + befehl + "'" + ', stdout=' + flag_op + ', stderr=' + flag_err + ', shell = True' + ', text = True'

command = Befehl_Schreiber()
print(command)
process = run(command)

the outcome as well as errors from code above in cmd is as below:

'usearch11 -threads 8 -fastq_filter .\Rohrdaten\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001.fastq -fastq_maxee 1 -fastq_minlen 120 -fastq_maxns 0 -fastaout .\1_QC\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001--1.QC.fasta', stdout=DEVNULL, stderr=PIPE, shell = True, text = True
Traceback (most recent call last):
  File "Fragestellung.py", line 13, in <module>
    process = run(command)
  File "C:\Program Files\Python38\lib\subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Program Files\Python38\lib\subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Program Files\Python38\lib\subprocess.py", line 1307, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] Das System kann die angegebene Datei nicht finden

when I copy the first line of the output above which is result of print(command) function in first code block into the place where 'command' was, the script is again running without problem:

from subprocess import run, DEVNULL, PIPE

def Befehl_Schreiber():
    flag_op = 'DEVNULL'
    flag_err = 'PIPE'
    befehl = 'usearch11 -threads 8 -fastq_filter ' + '.\Rohrdaten\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001.fastq' + ' -fastq_maxee 1 -fastq_minlen 120 -fastq_maxns 0 -fastaout ' + '.\\1_QC\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001--1.QC.fasta'
    return "'" + befehl + "'" + ', stdout=' + flag_op + ', stderr=' + flag_err + ', shell = True' + ', text = True'

process = run('usearch11 -threads 8 -fastq_filter .\Rohrdaten\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001.fastq -fastq_maxee 1 -fastq_minlen 120 -fastq_maxns 0 -fastaout .\1_QC\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001--1.QC.fasta', stdout=DEVNULL, stderr=PIPE, shell = True, text = True)

I could not figure out where the problem lies in...any help will be appreciated.

piwai · Answer 1 · 2022-03-25T12:49:02.490

I think the problem is that you're returning the extra parameters stdout/stderr/shell/string as a single string, which is passed directly to subprocess.run. You should try to return only the command with its arguments (the "befehl" variable) and include subprocess.run arguments only when calling the function.

EDIT: as detailled in answer below, if you really want to handle run args in the first method, you may add arguments in a separate dict, like this:

from subprocess import run, DEVNULL, PIPE

def Writer():
    flag_op = PIPE
    flag_err = PIPE
    cmd = 'ls /tmp'
    run_args = { 'shell': True, 'stdout': flag_op, 'stderr': flag_err }
    return cmd, run_args

cmd, args = Writer()
print(cmd)
run(cmd, **args)

Many thanks for both answers, now I understand wehre is the problem. Since I actually write a loop about the function 'Befehl' and each step of the loop may refers to different arguements(either DEVNULL or PIPE for stderr/stdout), I will take the solution which is detailled in the block.:) — citron, Mar 25 '22 at 13:21

tripleee · Answer 2 · 2022-03-25T13:30:10.190

The arguments to run need to be passed as arguments, not as part of the command string.

Compare how

print('hello, end=""')

and

print('hello', end="")

are different.

I can see no reason to break the command definition into a separate function. Here's a straightforward fix:

from subprocess import run, DEVNULL, PIPE

subprocess.run(
    ['usearch11', '-threads', '8',
     '-fastq_filter',  r'.\Rohrdaten\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001.fastq',
     '-fastq_maxee', '1', '-fastq_minlen', '120', '-fastq_maxns', '0',
     '-fastaout', r'.\\1_QC\pmFLP135_143_148_KWB3_1_1_5_S1_L001_R1_001--1.QC.fasta'],
    stdout=flag_op, stderr=flag_err, text=True, check=True)

with the added bonus of avoiding the pesky shell=True and adding check=True to ensure that the command completes successfully, as well as some quoting fixes.

If you really wanted to use a function to build the command and its flags, you can have it do something like

return (['usearch11', '-threads', '8', ...], {'stdout': flag_op, 'stderr': flag_op, ...})

and have the caller perform

subprocess.run(funcresult[0], **funcresult[1])

but this seems inordinately complex. Perhaps a suitable compromise if you really need to build the arguments to subprocess.run programmatically would be to have the function build just the command line, and have the caller hard-code the keyword arguments.

FileNotFound Error while runing subprocess.run in Python

2 Answers2