2

I'm currently launching a shell file on Gromacs (Linux) with 60 different commands in it. When launching the file the commands are processed sequentially one by one.

I want to launch all of them in parallel and in background.
How can I achieve that?

Here some lines of my shell file:

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_B.xvg -tu ns << eof
29
29
eof

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_C.xvg -tu ns << 
eof
32
32
eof

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_D.xvg -tu ns << 
eof
35
35
eof
hek2mgl
  • 152,036
  • 28
  • 249
  • 266
philDDD
  • 23
  • 2

3 Answers3

6

If you wanted to have the full benefit of a tool explicitly written for this type of thing, you could use GNU Parallel. You would probably change your shell script (let's call it commands) to look like this:

printf "29\n29\n" | gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_B.xvg -tu ns
printf "32\n32\n" | gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_C.xvg -tu ns

Then you could run:

parallel -j 8 < commands

if you wanted to run 8 at a time, or -j 32 if you wanted to run 32 at a time. You can test what it would do, without actually doing anything, by using --dry-run:

parallel --dry-run < cmds
printf "29\n29\n" | gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_B.xvg -tu ns
printf "32\n32\n" | gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_C.xvg -tu ns

You can also get an Estimated Time of Arrival with --eta, and also spread the jobs over any other computers on your network that you can ssh into. You get the idea - very flexible.

Mark Setchell
  • 191,897
  • 31
  • 273
  • 432
1

run them in the background using &, e.g. 

sleep 60 &
sleep 60 &
sleep 60 &

will run 3 times sleep in background (and they are executed in parallel). See e.g. http://www.cyberciti.biz/faq/linux-command-line-run-in-background/ or executing shell command in background from script

For the combination in use with the here_doc : Running script with HERE_DOC method in background . This should work (only one command shown):

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_B.xvg -tu ns << eof &
29
29
eof
Community
  • 1
  • 1
Raphael Roth
  • 26,751
  • 15
  • 88
  • 145
0

Based on the command you have written you might want to run something like (test all A..Z with all 1..100):

gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_{A..Z}.xvg -tu ns << 
eof
{1..100}
eof

Using GNU Parallel you would do this:

doit() {
  printf "$2\n$2\n" |
  gmx rms -s md.tpr -n index_analysis.ndx -f md.xtc -o rmsd_fc_chain_${1}.xvg -tu ns
}
export -f doit
parallel doit ::: {A..Z} ::: {1..100}

Learn more about GNU Parallel by walking through the tutorial: https://www.gnu.org/software/parallel/parallel_tutorial.html

Ole Tange
  • 31,768
  • 5
  • 86
  • 104