112

I would like to introduce multithreading feature in my shell script.

I have a script which calls the function read_cfg() with different arguments. Each of these function calls are independent.

Would it be possible to instantiate these function calls (not scripts) parallelly. Please let me how can we achieve that.. ?

CaptJak
  • 3,592
  • 1
  • 29
  • 50
Kiran
  • 8,034
  • 36
  • 110
  • 176
  • 2
    This is not multithreading -- it's multiprocessing. Each instance is run in a distinct process, copied from the original with `fork()`. These processes -- unlike threads -- have their own file descriptor tables, and their memory is copy-on-write (so when they change a variable's value, the parent process doesn't see it). – Charles Duffy Sep 21 '16 at 22:48

3 Answers3

236

Sure, just add & after the command:

read_cfg cfgA &
read_cfg cfgB &
read_cfg cfgC &
wait

all those jobs will then run in the background simultaneously. The optional wait command will then wait for all the jobs to finish.

Each command will run in a separate process, so it's technically not "multithreading", but I believe it solves your problem.

Martin
  • 37,119
  • 15
  • 73
  • 82
  • 12
    You should read up the difference between process and thread. What you propose is not multithreading - it involves separate processes for every command. – TomTom Mar 11 '10 at 14:56
  • 61
    @TomTom: I certainly know the difference between processes and threads. If you see through the OP's choice of words, I believe he is simply asking whether it's possible to run the commands in parallel (which is possible). I added a note about this to clarify. – Martin Mar 11 '10 at 14:59
  • 1
    It can be slow if you have an high amount of process – alexandre-rousseau Apr 19 '18 at 09:17
38

You can run several copies of your script in parallel, each copy for different input data, e.g. to process all *.cfg files on 4 cores:

    ls *.cfg | xargs -P 4 -n 1 read_cfg.sh

The read_cfg.sh script takes just one parameters (as enforced by -n)

tnorgd
  • 1,580
  • 2
  • 14
  • 24
  • 3
    just a note that you should specify the full path to `read_cfg.sh` or `xargs` will say it can't find the file. – Jeshurun Jul 16 '13 at 10:22
  • 3
    Better to use `printf '%s\0' *.cfg | xargs -0 ...` -- that way this works with filenames with spaces, unprintable characters, etc. See also [Why you shouldn't parse the output of ls(1)](http://mywiki.wooledge.org/ParsingLs). – Charles Duffy Sep 21 '16 at 22:49
28

Bash job control involves multiple processes, not multiple threads.

You can execute a command in background with the & suffix.

You can wait for completion of a background command with the wait command.

You can execute multiple commands in parallel by separating them with |. This provides also a synchronization mechanism, since stdout of a command at left of | is connected to stdin of command at right.

mouviciel
  • 66,855
  • 13
  • 106
  • 140