2

I have a python script which I normally execute from BASH shell like this:

pychimera $(which dockprep.py) -rec receptor1.pdb -lig ligand1.mol -cmethod gas -neut

As you see some of the arguments need an input (e.g. -rec) while others don't (e.g. -neut). I must execute this script 154 times with different inputs. How is it possible to run 8 threads in parallel using GNU parallel script?

pychimera $(which dockprep.py) -rec receptor1.pdb -lig ligand1.mol -cmethod gas -neut
pychimera $(which dockprep.py) -rec receptor2.pdb -lig ligand2.mol -cmethod gas -neut
pychimera $(which dockprep.py) -rec receptor3.pdb -lig ligand3.mol -cmethod gas -neut
...
tevang
  • 518
  • 1
  • 4
  • 17
  • You need to create a file with all commands in it. You can build that file automatically (and we can help with enough information to build that file). Then `module load parallel && parallel < commands.txt` –  Apr 06 '19 at 12:29

2 Answers2

3

I think you want this:

parallel 'pychimera $(which dockprep.py) -rec receptor{}.pdb -lig ligand{}.mol -cmethod gas -neut' ::: {1..154}

If you have other than 8 CPU cores, and specifically want 8 processes at a time, use:

parallel -j8 ...

If you want to see the commands that would be run without actually running anything, use:

parallel --dry-run ...
Mark Setchell
  • 191,897
  • 31
  • 273
  • 432
  • The --dry-run is very informative. But the input files do not necessarily follow the simplified format with indices that I used. They may vary completely. – tevang Apr 06 '19 at 19:48
  • The command I suggested works exactly the same as the answer you have accepted! I can only help you as well as your question describes your problem - if your problem is harder please alter your question and I, or maybe Ole (the author of **GNU Parallel**) will assist further. – Mark Setchell Apr 06 '19 at 20:04
  • @MarkSetchell I cannot help further: Your answer is literally character for character what I would have written. – Ole Tange Apr 07 '19 at 20:07
1

Example commands.txt generator script:

#!/usr/bin/env bash

if [ "$#" -ne 1 ]; then
    echo "missing parameter: n"
    exit 1
fi

rm commands.txt 2> /dev/null 
dockp=$(which dockprep.py)

for((i=1;i<=$1;i++)); do
  echo "pychimera $dockp -rec receptor$i.pdb -lig ligand$i.mol -cmethod gas -neut" >> commands.txt
done

If you save above bash script as cmdgen.sh you can run it as:

bash cmdgen.sh 100

if you need n to be 100.

To run commands in parallel:

$ module load parallel
$ parallel < commands.txt
  • Yes, it works great! I can control the number of threads like this "parallel -j8 < commands.txt". – tevang Apr 06 '19 at 19:41