1

I was able to install GNU Parallel globally in git-bash by following this answer.

However, on running an example command as mentioned in the parallel-tutorial,

parallel -k echo ::: A B C > abc-file

I keep getting this error 

sh: -c: option requires an argument
sh: -c: option requires an argument
sh: -c: option requires an argument
.
.
.

What am I doing wrong here?

Saurabh P Bhandari
  • 6,014
  • 1
  • 19
  • 50

2 Answers2

1

So the root cause is that CygWin (contrary to GNU/Linux) does not respect redirection of STDERR if the command line is too long.

GNU Parallel figures out how long the longest possible command line is by doing a binary search for the length. This is awfully slow on CygWin because forking a 12 MB command line is horribly slow (and 12 MB seems to be the limit in my version of CygWin).

Luckily it only has do be done once. After this GNU Parallel caches the line length in ~/.parallel/tmp/HOSTNAME/linelen, and this is the reason why you experience the problem when ~/.parallel/tmp is removed.

This is also the reason why it seemed that using a different version worked: You simply had a single run that finished, and thus cached the length. It was not the change of version that did this.

Until I manage to get CygWin to ignore the sh: -c: option requires an argument all you need to do is to ignore it and be patient. I should probably also put in a small warning, to let CygWin users know that they have to be patient the first time.

Run:

parallel echo ::: 1

It will spit out the sh: -c: option requires an argument around 25 times, but that is fine. It will take around 30 seconds to complete.

After this everything should be fast(er) and you should not see the error.

It should be fixed in the newest version in GIT: https://savannah.gnu.org/git/?group=parallel

Ole Tange
  • 31,768
  • 5
  • 86
  • 104
0

You are not the first to have this problem, and currently we do not know what causes it. I have access to a Windows-10 machine and I do not see that behaviour. A workaround seems to be using an older version of GNU Parallel. You can help by figuring out which versions work. When you have a single version (look here: https://ftpmirror.gnu.org/parallel), that works, run this:

testone() {
  v="$1"
  wget -c https://ftpmirror.gnu.org/parallel/parallel-$v.tar.bz2
  tar xvf parallel-$v.tar.bz2
  cd parallel-$v
  src/parallel true ::: 1
}
export -f testone
parallel -k --joblog my.log testone {1}{2}22 ::: {2012..2020} ::: {01..12}
grep -E '\s0\s0\stest' my.log

This will give all versions that do work.

Post the output from parallel -Dall echo ::: foo for both the newest working version, the following version, and the newest version (20200322).

Ole Tange
  • 31,768
  • 5
  • 86
  • 104
  • Thank you for your answer, I got it to work with version 20181022 but the newest version which was installed globally also started working after this. It worked only when one of the older versions didn't give this error `sh: -c: option requires an argument`, for ex: this version 20191022 gave the error and the global install gave the same error – Saurabh P Bhandari Mar 23 '20 at 00:19
  • One more observation, 20181022 version creates a `tmp` dir in `.parallel` dir. For version 20200322, there was no `tmp` dir in `.parallel` dir. So before running 20181022 version, 20200322 version returns the error and after running 20181022 version (which returns no error) , the 20200322 version starts working and a `tmp` dir exists in `.parallel` dir (`tmp` dir was created by the 20181022 version). To test if `tmp` dir is involved in the running of the 20200322 version, I deleted the dir and it again started returning the error – Saurabh P Bhandari Mar 23 '20 at 00:39
  • The above code snippet returns versions which are not working too, also since parallel command run here `parallel -k --joblog my.log testone {1}{2}22 ::: {2012..2020} ::: {01..12}` causes creation of `tmp` dir in `.parallel` dir, probably executing this `rm -rf ~/.parallel/tmp` in the testone function and limiting the jobs to 1 (`-j1`) in the parallel command after export should do it, but still it returns all versions – Saurabh P Bhandari Mar 23 '20 at 02:55
  • 1
    Excellent debugging. I can now reproduce the error. So it clearly has something to do with .parallel/tmp. Expect a fix in next release. – Ole Tange Mar 23 '20 at 08:21