Run a bash command in parallel with xargs

Question

I hope someone can help me with this simple problem. I want to run this command in parallel

windmill chrome test=./test http://www.google.ch

I was playing around with xargs and looked at the examples in the internet. However, I was not able to construct a xargs command to execute my mentioned command in parallel.

I tried the following

echo "chrome test=./test http://www.google.ch" | xargs -n 1 -P 2 windmill

which did not execute the right command meaning that windmill must have executed the wrong command because the output is not correct (default output of windmill for specifying wrong arguments).

Nevertheless, I got then another problem namely that the terminal/python complained about "socket.error: [Errno 48] Address already in use".

So when I run the windmill command in parallel by just openen for example two terminals and run in each terminal the windmill command and it works.

If xargs is not solution then I appreciate it a lot if you can pinpoint me to the correct way how to do it :)

score 1 · Answer 1 · answered Dec 11 '11 at 09:03

1

xargs would not run your commands is parallel.

Why don't you just execute this in any bourne-compatible shell?

for i in 1 2; do windmill chrome test=./test http://www.google.ch & done

answered Dec 11 '11 at 09:03

Alan

504
2
5

This works well. However, I get a socket binding error. It seems that the commands must somehow being ran in different terminals to have separate sockets? – mkn Dec 11 '11 at 09:10
No, sockets have nothing in common with terminals. Seems that your program just can not run multiple instances cause it is trying to bind the same socket. Maybe there is a command-line parameter to change the binding address/port ? – Alan Dec 11 '11 at 09:20
1

Hm but why is it then possible to run the command in different terminals at the same time? I have look at the specification of windmill if I can set the sockets manually (there is not really a proper documentation >:[) – mkn Dec 11 '11 at 13:35
I've found that sometimes a program will have a problem binding to a recently released socket, and sometimes it won't. – dstromberg Dec 12 '11 at 08:20
`xargs` most certainly can run commands in parallel with the `-P` option. It's right there on the man page. – John Dec 10 '22 at 15:09

score 1 · Answer 2 · edited Feb 26 '14 at 07:59

You mention you want to run a command in parallel. That can only be done if the program itself is parallelized internally.

What you can do, however, is run multiple commands in parallel. Say you want to run these in parallel:

windmill chrome test=./test http://www.google.ch
windmill chrome test=./test http://www.google.se
windmill chrome test=./test http://www.google.no
windmill chrome test=./test http://www.google.de

Using GNU Parallel you can do:

parallel windmill chrome test=./test http://www.google.{} ::: ch se no de

Watch the intro videos to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1 and walk through the tutorial (man parallel_tutorial). You command line with love you for it.

If that is not what you want, please re-phrase your question.

score 0 · Answer 3 · answered Dec 12 '11 at 08:21

I've found that sometimes a program will have a problem binding to a recently released socket - the kernel tries to prevent this for security reasons. If you need to be able to rebind immediately, you probably should try SO_REUSEADDR or just bind to a different port each time.

Run a bash command in parallel with xargs

3 Answers3