xargs input involving spaces

Question

I am working on a Mac using OSX and I'm using bash as my shell. I have a script that goes something to the effect of:

VAR1="pass me into parallel please!"
VAR2="oh me too, and there's actually a lot of us, but its best we stay here too"
printf "%s\n" {0..249} | xargs -0 -P 8 -n 1 . ./parallel.sh

I get the error: xargs: .: Permission denied. The purpose is to run a another script in parallel (called parallel.sh) which get's fed the numbers 0-249. Additionally I want to make sure that parallel can see and us VAR1 and VAR2. But when I try to source the script parallel with . ./parallel, xargs doesn't like that. The point of sourcing is because the script has other variables I wish parallel to have access to.

I have read something about using print0 since xargs separates it's inputs by spaces, but I really didn't understand what -print0 does and how to use it. Thanks for any help you guys can offer.

What do you mean by "running in parallel" and "parallel access"? You want multiple processes running the parallel.sh script and communicating through shared variables? — Vaughn Cato, Jun 06 '13 at 02:37
Isn't that how you make the variables declared in the script available to the parallel script? I thought that was how you "sourced". Keep in mind I'm a novice -at best. — Novice C, Jun 06 '13 at 02:38
@VaughnCato I rewrote the original post to try and explain better. But in short yes I want multiple processes of parallel.sh running, and I want them all to be able to have read the variables. — Novice C, Jun 06 '13 at 02:40

Vaughn Cato · Accepted Answer · 2013-06-06T13:36:54.857

2

If you want the several processes running the script, then they can't be part of the parent process and therefore they can't access the exact same variables. However, if you export your variables, then each process can get a copy of them:

export VAR1="pass me into parallel please!"
export VAR2="oh me too, and there's actually a lot of us, but its best we stay here too"
printf "%s\n" {0..249} | xargs -P 8 -n 1 ./parallel.sh

Now you can just drop the extra dot since you aren't sourcing the parallel.sh script, you are just running it.

Also there is no need to use -0 since your input is just a series of numbers, one on each line.

edited Jun 06 '13 at 13:36

answered Jun 06 '13 at 02:51

Vaughn Cato

63,448
5
82
132

1

`bash` doesn't have an `extern` keyword; I think you mean `export`. – chepner Jun 06 '13 at 12:30
@chepner: geez. thanks! writing too much C code today... :) – Vaughn Cato Jun 06 '13 at 13:37
Isn't using `xargs` to run `parallel` rather ... overkill? From the `parallel` man page: _If you use `xargs` today you will find GNU `parallel` very easy to use as GNU `parallel` is written to have the same options as `xargs`_ – Jonathan Leffler Jan 18 '14 at 21:48

score 2 · Answer 2 · answered Aug 03 '16 at 11:48

2

To avoid the space problem I'd use new line character as separator for xargs with the -d option:

xargs -d '\n' ...

answered Aug 03 '16 at 11:48

Ray

5,269
1
21
12

mgamba · Answer 3 · 2013-06-06T15:18:09.900

1

The issue of passing arguments is related to xarg's interpretation of white space. From the xargs man page:

-0      Change xargs to expect NUL (``\0'') characters as separators, instead of spaces and newlines.

The issue of environment variables can be solved by using export to make the variables available to subprocesses:

say.sh

echo "$1 $V"

result

bash$ export V=whatevs
bash$ printf "%s\n" {0..3} | xargs -P 8 -n 1 ./say.sh
1 whatevs
2 whatevs
0 whatevs
3 whatevs

edited Jun 06 '13 at 15:18

answered Jun 06 '13 at 02:28

mgamba

1,189
8
10

That doesn't allow me to use the variables I've created in this script though. – Novice C Jun 06 '13 at 02:39
Removing the `-0` argument allows xargs to use `\n` as a separator. – mgamba Jun 06 '13 at 02:53
@NoviceC You need to export the variables, since running `parallel.sh` via `xargs` precludes running it in the same shell as where the variables are defined. – chepner Jun 06 '13 at 12:30
Ah, sorry, I didn't see that this was a two-part question. – mgamba Jun 06 '13 at 15:09

score 1 · Answer 4 · edited May 23 '17 at 12:00

i think you have permission issues , try getting a execute permission for that file "parallel.sh"

command works fine for me :

Kaizen ~/so_test $ printf "%s\n" {0..4} | xargs -0 -P 8 -n 1 echo
0
1
2
3
4

man find :

-print0

True; print the full file name on the standard output, followed by a null character (instead of the newline character that -print uses). This allows file names that contain newlines or other types of white space to be correctly interpreted by programs that process the find output. This option corresponds to the -0 option of xargs.

for print0 use : check the link out : there is a question for it in stack overflow

Capturing output of find . -print0 into a bash array

xargs input involving spaces

4 Answers4

Linked