bash gnu parallel argfile syntax

Question

I just discovered GNU parallel and I'm having some trouble running a simple parallel task. I have a simulation running over multiple values and I'd like to split it up to run in parallel using command line args. From the docs , it seems you can run parallel mycommand :::: myargfile in which myargfile contains the various arguments you would like to feed your command, in parallel. However, I didn't see any information on how the args should be listed and assumed a myargfile like this would work:

--pmin 0 --pmax 0.1
--pmin 0.1 --pmax 0.2
...

mycommand --pmin 0 --pmax 0.1 executes no problem. But when I run parallel mycommand :::: myargfile I get error: unknown option pmin 0 --pmax 0.1 (caught and decoded courtesy boost program options). parallel echo :::: myargfile correctly prints out the arguments. It's as if they are being wrapped in a string which the program can't read and not fed like they are from a standard bash script.

What's going on? How can I make this work?

This question might have been answered [here](http://stackoverflow.com/a/6258206/1328439). — Dima Chubarov, Jan 20 '13 at 20:27

score 1 · Accepted Answer · edited May 23 '17 at 12:01

1

Following @DmitriChubarov's link to https://stackoverflow.com/a/6258206/1328439 , I discovered that I was lacking the colsep flag:

parallel --colsep ' ' mycommand :::: myargfile

successfully executes.

edited May 23 '17 at 12:01

Community

1
1

answered Jan 20 '13 at 20:47

mmdanziger

4,466
2
31
47

score 0 · Answer 2 · answered Sep 07 '16 at 13:23

After digging through manual and help pages I came up with this example. Perhaps it will save someone out there. :)

#!/usr/bin/env bash

COMMANDS=(
  "cnn -a mode=flat"
  "cnn -a mode=xxx"
  "cnn_x -a mode=extreme"
)

parallel --verbose --progress --colsep ' ' scrapy crawl {.} ::: "${COMMANDS[@]}"

bash gnu parallel argfile syntax

2 Answers2