1

I have a module that I want to run on every file in a directory. However, when I iterate over that directory, using each file as an input, the module cannot find the file, as though the variable defined in the loop doesn't actually point to the file. Here is the code I am trying to execute:

import os as os

for file in os.listdir():
    if file.endswith('.fasta'):   
        !python ../iupred2a.py file long

Any help is greatly appreciated. Thanks!

Adam Smith
  • 52,157
  • 12
  • 73
  • 112
MParker
  • 11
  • 1
  • Sorry, the last line of code should have as input "file", not "C9QPE2.fasta – MParker Feb 22 '19 at 19:27
  • How do you know the module can't find the file? Is it giving you an error message like "can't find file"? Because the error message your code is giving me is `SyntaxError: invalid syntax`. Syntax errors usually do not happen because of a missing file. – Kevin Feb 22 '19 at 19:29
  • Yes, the module throws the error message: "Input sequence file not found at file!" – MParker Feb 22 '19 at 19:30
  • 3
    Is this jupyter? What's the `!python` notation? – Adam Smith Feb 22 '19 at 19:31
  • Based on the error message, you're passing the literal string `file` to the program, NOT the variable holding the filename. This isn't really a Python question at all, it's specific to whatever IDE is providing that `!python` syntax. – jasonharper Feb 22 '19 at 19:34
  • Yes, this is Jupiter. – MParker Feb 22 '19 at 19:35
  • Agreed, it is passing the literal string 'file' instead of the associated value within the loop to module. Is there anyway around this or do I have to make a change within the module itself? – MParker Feb 22 '19 at 19:39
  • 1
    I don't know much about jupyter, but I'm guessing that `!python ../iupred2a.py file long` locates a file named `iupred2a.py` and executes it with the command line paramters "file" and "long". Putting aside the matter of whether this works, it's not very idiomatic. The conventional way to access the functionality of another Python program is to import it and call its functions normally. If you do this, then you won't have to worry about how Jupyter interprets your `file` argument. – Kevin Feb 22 '19 at 19:40

1 Answers1

1

https://ipython.readthedocs.io/en/stable/interactive/reference.html#system-shell-access says that shell commands (for example, your line prefixed with a "!") are interpreted literally. When you type "file", it sees "file", not the value of your file variable.

Any input line beginning with a ! character is passed verbatim (minus the !, of course) to the underlying operating system.

But it also says you can use braces or a dollar sign to "expand" a value.

IPython also allows you to expand the value of python variables when making system calls. Wrap variables or expressions in {braces}:

In [1]: pyvar = 'Hello world'
In [2]: !echo "A python variable: {pyvar}"
A python variable: Hello world
In [3]: import math
In [4]: x = 8
In [5]: !echo {math.factorial(x)}
40320

For simple cases, you can alternatively prepend $ to a variable name:

In [6]: !echo $sys.argv
[/home/fperez/usr/bin/ipython]
In [7]: !echo "A system variable: $$HOME"  # Use $$ for literal $
A system variable: /home/fperez

In your case, try !python ../iupred2a.py $file long or !python ../iupred2a.py {file} long.

... All that said, I think it would be better to just import your other Python file and call its functions directly. This may require a little redesigning, because importing from a file from one directory up is somewhat tricky, and the command-line interface for a module is usually different from its programming interface.

If you can get your current file and iupred2a.py into the same directory, and figure out the name of the function that you actually want to call, then your code would end up looking something like:

import os
import iupred2a as iup

for file in os.listdir():
    if file.endswith('.fasta'):   
        iup.do_the_thing(file, mode="long")
Kevin
  • 74,910
  • 12
  • 133
  • 166