13

I want to do the following things:

  • Execute multiple shell scripts (here 2 scripts) concurrently.

  • Wait until both scripts finish

  • Dump return value of each script

However, main.sh does not work as expected.


main.sh

#!/bin/bash

ret1=`./a.sh` &
ret2=`./b.sh`

if [ "${ret1}"="" -a "${ret2}"="" ]; then
   sleep 1
else
   echo ${ret1},${ret2}
end

a.sh

#!/bin/bash
sleep 10
echo 1

b.sh

#!/bin/bash
sleep 5
echo 2
Community
  • 1
  • 1
akry
  • 615
  • 2
  • 7
  • 14
  • Take a look at http://stackoverflow.com/questions/356100/how-to-wait-in-bash-for-several-subprocesses-to-finish-and-return-exit-code-0 – nimrodm Nov 23 '11 at 17:11

5 Answers5

12

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

parallel -j0 '{}; echo $?' ::: a.sh b.sh

I have a suspicion that you want the exit code to check if one of them failed, and that you actually do not care what the precise exit code was. In that case you can do:

parallel -j0 ::: a.sh b.sh || echo one or both of them failed

If it is sufficient to get the error code of the last that failed:

parallel -j0 --halt 1 ::: a.sh b.sh; echo $?

Maybe you would like to kill a.sh if b.sh finishes early but fails:

parallel -j0 --halt 2 ::: a.sh b.sh; echo $?

You can install GNU Parallel simply by:

$ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
   fetch -o - http://pi.dk/3 ) > install.sh
$ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
12345678 883c667e 01eed62f 975ad28b 6d50e22a
$ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
cc21b4c9 43fd03e9 3ae1ae49 e28573c0
$ sha512sum install.sh | grep da012ec113b49a54e705f86d51e784ebced224fdf
79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
$ bash install.sh

Watch the intro videos for GNU Parallel to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Print the cheat sheet: https://www.gnu.org/software/parallel/parallel_cheat.pdf

Ole Tange
  • 31,768
  • 5
  • 86
  • 104
  • 1
    +1 for GNU parallel. At first I tried using `wait` as suggested by Lee Netherton and others. And I immediately found myself thinking about how to write code to collect exit statuses in a list, loop over them, etc. GNU parallel takes care of all that logistical crap :) right out of the box. It was as simple as `find . -name *.sh | parallel -j0 sh {}` – Noah Sussman Oct 21 '12 at 17:03
8

Here is some code that I have been running, that seems to do exactly what you want. Just insert ./a.sh and ./b.sh where appropriate:

# Start the processes in parallel...
./script1.sh 1>/dev/null 2>&1 &
pid1=$!
./script2.sh 1>/dev/null 2>&1 &
pid2=$!
./script3.sh 1>/dev/null 2>&1 &
pid3=$!
./script4.sh 1>/dev/null 2>&1 &
pid4=$!

# Wait for processes to finish...
echo -ne "Commands sent... "
wait $pid1
err1=$?
wait $pid2
err2=$?
wait $pid3
err3=$?
wait $pid4
err4=$?

# Do something useful with the return codes...
if [ $err1 -eq 0 -a $err2 -eq 0 -a $err3 -eq 0 -a $err4 -eq 0 ]
then
    echo "pass"
else
    echo "fail"
fi

Note that this captures the exit status of the script and not what it outputs to stdout. There is no easy way of capturing the stdout of a script running in the background, so I would advise you to use the exit status to return information to the calling process.

Lee Netherton
  • 21,347
  • 12
  • 68
  • 102
4

The answer you seek is in this question shell - get exit code of background process

Basically, when you background a process you can't get its exit code directly. But if you run the bash wait command, then wait's exit code will return the exit code of the background process.

./a.sh &
pid1=$!
./b.sh
ret2=$?
wait ${pid1}
ret1=$?

This will work even if a.sh ends before you run wait. The special variable $? holds the exit code of the previous process. And $! holds the Process ID of the previously run process.

Community
  • 1
  • 1
Michael Dillon
  • 31,973
  • 6
  • 70
  • 106
  • This will not put the return value of wait in ret1, but will always set ret1 to the empty string. (wait does not write anything to stdout) – William Pursell Nov 23 '11 at 20:23
2

If you have bash 4.2 or later available the following might be useful to you. It uses associative arrays to store task names and their "code" as well as task names and their pids. I have also built in a simple rate-limiting method which might come handy if your tasks consume a lot of CPU or I/O time and you want to limit the number of concurrent tasks.

The script launches all tasks in the first loop and consumes the results in the second one.

This is a bit overkill for simple cases but it allows for pretty neat stuff. For example one can store error messages for each task in another associative array and print them after everything has settled down.

(I have copied this answer over from my answer here because it solves both questions, if that's not ok please tell me or replace it directly with just a link or whatever is suitable.)

#! /bin/bash

main () {
    local -A pids=()
    local -A tasks=([task1]="echo 1"
                    [task2]="echo 2"
                    [task3]="echo 3"
                    [task4]="false"
                    [task5]="echo 5"
                    [task6]="false")
    local max_concurrent_tasks=2

    for key in "${!tasks[@]}"; do
        while [ $(jobs 2>&1 | grep -c Running) -ge "$max_concurrent_tasks" ]; do
            sleep 1 # gnu sleep allows floating point here...
        done
        ${tasks[$key]} &
        pids+=(["$key"]="$!")
    done

    errors=0
    for key in "${!tasks[@]}"; do
        pid=${pids[$key]}
        local cur_ret=0
        if [ -z "$pid" ]; then
            echo "No Job ID known for the $key process" # should never happen
            cur_ret=1
        else
            wait $pid
            cur_ret=$?
        fi
        if [ "$cur_ret" -ne 0 ]; then
            errors=$(($errors + 1))
            echo "$key (${tasks[$key]}) failed."
        fi
    done

    return $errors
}

main
Community
  • 1
  • 1
stefanct
  • 2,503
  • 1
  • 28
  • 32
  • Hi Sorry for jumping in late but i want to capture start time end time and elapsed time for each individual task completed/incomplete. But i am unable to do so – Vikram Singh Chandel Oct 07 '15 at 12:54
  • You could add another array similar to `pids` and store the begin time in there but it would require a different style of waiting for the processes. ENOTIME to figure something out, maybe creating a new question might be better anyway? – stefanct Oct 08 '15 at 10:08
  • Hi Just need one clarification can you explain this line jobs 2>&1 How does it only get the jobs run my me at present? What will happen if some other jobs are also running which are not triggered by me(user) – Vikram Singh Chandel Oct 12 '15 at 12:17
  • 1
    `jobs` prints only the shell jobs of the current shell. It does not include jobs from other shells/terminals or even other users. However, if you start a background job before running a script using the code above this job would be returned as well and be part of the number of "running" jobs. That might or might not be wanted but I guess it makes sense to include it in most cases. Else one would have to record the jobs that exist when the script is started and filter them out for the calculation. – stefanct Oct 12 '15 at 15:16
0

Backticks do not give the value returned by the command, but the output of the command. To get the return values:

#!/bin/sh

./a.sh &
./b.sh
ret2=$?   # get value returned by b.sh
wait %1   # Wait for a.sh to finish
ret1=$?   # get value returned by a.sh
echo "$ret1: $ret2"

If you mistated the question and do in fact want the output of the commands, you get that as well since they will both go to the stdout of the script.

William Pursell
  • 204,365
  • 48
  • 270
  • 300