2

I'm trying to execute a python file in SunGrid engine, and I'm executing it from my anaconda3 environment variable.

my code is simple:

from __future__ import print_function
import urllib3
import numpy as np

if __name__ == '__main__':
    print('Hellooo')

I'm calling it like: 

qsub -V -b n -cwd -pe mp 3 playground.py

but I am getting this error:

from: can't read /var/mail/__future__
import: unable to open X server `' @ error/import.c/ImportImageCommand/358.
/var/spool/gridengine/execd/cluster-rp-02/job_scripts/22924: 3: /var/spool/gridengine/execd/cluster-rp-02/job_scripts/22924: Syntax error: word unexpected (expecting ")")

I looked online for the error and I found a solution her: Getting Python error “from: can't read /var/mail/Bio”

it proposed to add: #!/usr/bin/env python in the beginning of the python code.

I'm using anaconda3 where the destination of the used python is not the same. So, it should be: #!../anaconda3/envs/py3/bin/python

But when I add this script I get this error:

/home/master/bin/sge_mp_startup.sh: 10: exec: /var/spool/gridengine/execd/cluster-rp-01/job_scripts/22926: not found

Did I miss something?

wjandrea
  • 28,235
  • 9
  • 60
  • 81
Minions
  • 5,104
  • 5
  • 50
  • 91
  • Documentation: [How to submit a job using qsub](http://bioinformatics.mdc-berlin.de/intro2UnixandSGE/sun_grid_engine_for_beginners/how_to_submit_a_job_using_qsub.html) and [`qsub` man page](http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html) – wjandrea Jul 03 '19 at 14:01
  • @wjandrea, nothing there related to this issue. – Minions Jul 03 '19 at 14:44
  • Not directly no, but they explain how `qsub` works, which is helpful for other readers. And you might find some useful info there yourself. – wjandrea Jul 03 '19 at 14:53
  • Why are you using `__future__`? You're using Python 3 and from what I understand [here](https://stackoverflow.com/questions/7075082/what-is-future-in-python-used-for-and-how-when-to-use-it-and-how-it-works), it's unnecessary and seems to be causing your error. – m13op22 Jul 03 '19 at 15:53
  • @HS-nebula , it was in my sample code before. Anyway, also after deleting it I get the same erro. – Minions Jul 03 '19 at 16:02
  • @HS-nebula `__future__` is irrelevant here. The actual problem is that the Python script is being interpreted by a shell. See the linked question. – wjandrea Jul 03 '19 at 20:33
  • 1
    @Ghanem I just noticed you didn't try putting the absolute path of the `python` executable in the shebang. Try that. – wjandrea Jul 03 '19 at 20:35
  • @wjandrea The first error looked like it had issues with import future, which is why I asked. And good comment, I just noticed that as well – m13op22 Jul 03 '19 at 20:39
  • @wjandrea , what do you mean by shebang? – Minions Jul 03 '19 at 21:06
  • @Ghanem the first line, which starts with `#!` – wjandrea Jul 03 '19 at 21:07
  • but this is my problem! I already mentioned that I added it like: `#!../anaconda3/envs/py3/bin/python`, or do you mean another thing? – Minions Jul 03 '19 at 21:10
  • @Ghanem Let me rephrase: try putting the absolute path of the `python` executable in the shebang. I.e. replace the `../`. – wjandrea Jul 04 '19 at 17:07
  • @wjandrea I tried it like: `#!/home/bghanem/anaconda3/envs/py3/bin/python` , but it returns the same error. – Minions Jul 04 '19 at 17:45

1 Answers1

0

From the linked question:

If your script is stored in a file named script.py, you have to execute it as python script.py

So you could add ../anaconda3/envs/py3/bin/python to the command line:

qsub -V -b n -cwd -pe mp 3 ../anaconda3/envs/py3/bin/python playground.py

Or if ../anaconda3/envs/py3/bin/python is the first python executable in your path, you could simplify:

qsub -V -b n -cwd -pe mp 3 python playground.py
wjandrea
  • 28,235
  • 9
  • 60
  • 81
  • for the first script: Unable to run job: `Script length does not match declared length Exiting`. And for the second: `Unable to read script file because of error: error opening python: No such file or directory.` – Minions Jul 03 '19 at 13:53
  • @Ghanem I don't have any experience with `qsub`, so I have no idea what it's doing. – wjandrea Jul 03 '19 at 13:56
  • @Ghanem maybe you need to remove the `-cwd` option? – wjandrea Jul 03 '19 at 13:57
  • @Ghanem Or switch `-b n` for `-b y`? – wjandrea Jul 03 '19 at 14:01
  • I tried to remove `-cwd` but it doesn't work. Also I tried to switch to `-b y`, I got: `/home/master/bin/sge_mp_startup.sh: 10: exec: playground.py: not found` – Minions Jul 03 '19 at 14:04