58

I have been trying to use GNU parallel for some time, but I have never been able to get it to function at all!

For example, running (in a non-empty directory!):

ls | parallel echo            # Outputs single new line
ls | parallel echo echo echo  # Outputs three new lines.
ls | parallel echo {}         # /bin/bash: {}: command not found
ls | parallel echo '{}'       # /bin/bash: {}: command not found
ls | parallel 'echo {}'       # Outputs: {}
ls | parallel -IMM 'echo MM'  # Outputs: MM

It seems that it is simply executing each argument as a command, which makes no sense.

I have tried bash, zsh, tcsh, csh, and sh, to no avail.

WaelJ
  • 2,942
  • 4
  • 22
  • 28

4 Answers4

91

As I was about to complete writing this question, I ran parallel --version to report the version, only to find:

WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.

It is not clear to me why that flag is set by default. Needless to say, using --gnu worked!

Thought I would post this to save someone hours of frustration and confusion.

EDIT: To fix this permanently (in Ubuntu at least), delete the --tollef flag in /etc/parallel/config

WaelJ
  • 2,942
  • 4
  • 22
  • 28
  • 22
    --tollef will be retired 20140222 http://lists.gnu.org/archive/html/parallel/2013-02/msg00018.html It will be helpful if you already now let your frustration be known to your distribution maintainer, so that the default can be changed. – Ole Tange May 10 '13 at 11:04
  • @Thor what do you mean by 'improving the situation'? As in changing the default behavior? – WaelJ May 10 '13 at 12:27
  • @OleTange That is very nice to hear! :) – WaelJ May 10 '13 at 12:28
  • Does anyone know what this actully does. I had this problem and the fix worked. – meawoppl Sep 12 '13 at 00:03
  • 1
    This bit me hard. I'm running Ubuntu, and Ubuntu does this by default to everybody. See Launchpad issue [here](https://bugs.launchpad.net/ubuntu/+source/parallel/+bug/1183963). – Alexander Garden Jan 25 '14 at 02:00
  • 2
    Just so everyone knows, @OleTange (1st comment above) is the author of Parallel! – Elijah Lynn Feb 09 '14 at 13:32
  • 1
    @ElijahLynn It makes it so much easier to predict stuff, when you can make them happen yourself. – Ole Tange Feb 09 '14 at 14:30
  • 2
    The problem exists and this solves the problem in Fedora 20 as well. Not being able to figure this out prevented me from using `parallel` until now! – alfC Aug 19 '14 at 20:37
17

Depending on your operating system, you should check whether you're actually running the GNU version.

$ parallel --version
parallel: invalid option -- '-'
parallel [OPTIONS] command -- arguments
    for each argument, run command with argument, in parallel
parallel [OPTIONS] -- commands
    run specified commands in parallel

If this is the case, you're not running the GNU version. Ubuntu 12.04 is like this, and you'll need to manually install GNU parallel to get the functionality you expect.

Community
  • 1
  • 1
Ian
  • 11,280
  • 3
  • 36
  • 58
  • 4
    Am using Debian 10 and this is still a problem. You have to manually install parallel, but until you do `parallel` is linked to some useless command that does nothing – UTF_or_Death Oct 09 '20 at 17:16
1

Had issues running parallel as an external command from FREEMAT (MATLAB lookalike); the argumentFile was not fed to the command properly solved it by:

  • Adding --gnu to options
  • Not using cmdString syntax involving ["]

Code:

cmdString = 'parallel --gnu command ::: ';
    while j<=jLength
        cmdString = [cmdString argumentFilePath(j,:) ' '];
        j=j+1;
    end
    system(cmdString)

Thank you for that :) Im on Ubuntu 12.04 as well.

Erik D
  • 11
  • 2
0

For me it was same issue but different problem. Just running parallel command was exiting silently. Also parallel --version was saying invalid option error. In my Path there was just one parallel executable binary but still it was not detecting.

I was able to fix it as below:

  1. Run whereis parallel. This gives all the paths where executables named parallel is present. For my case there was just one path /usr/local/bin/parallel. Running using this path works just fine.
  2. You can add an alias for this in ~/.bashrc or ~/.zshrc file like alias parallel='/usr/local/bin/parallel'

And now parallel works like charm.

dev-dsk % parallel --version         
GNU parallel 20190322
Copyright (C) 2007-2019 Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.
bit_cracker007
  • 2,341
  • 1
  • 26
  • 26