0

Have six scripts which I want to execute once a day using following logic:
s11, s12 can start and run parallel
s21, s22 should only start after s11 and s12 finished. Both can run parallel
s31, s32 should only start after s21 and s22 finished. Both can run parallel

So far I did it by starting a daily masterscript m by cron. m started all six scripts s11-s32, s11 and s12 did their job directly but the others looked every minute in a counter file and only if the counter had the right value they started the real job. Each script changed the counter before closing, this was the handover to the next script generation. But for other reasons my server was too busy that the new cron started m before the yesterday scripts finished and I screwed up my data.

I assume others had similar problems and know a little library or anything else to get this done properly and stable, for sure the new series shouldn't start before the old finished... .

Thanks in advance for any hints!

tempo
  • 103
  • 3

1 Answers1

2

A example of master script in bash , i used wait to wait for the completion of
script that was started in background with &

This example assume : that all yours scripts are a folder /home/me/myproject/ you have a logs folder where you want to capture some outputs

#!/bin/bash

cd /home/me/myproject/   

bin/s11 > logs/s11_stdout.log 2>logs/s11_stderr.log   &
bin/s12 > logs/s12_stdout.log 2>logs/s12_stderr.log   &
wait


./s21 &
./s22 &
wait

s31 &
s32 &
wait
EchoMike444
  • 1,513
  • 1
  • 9
  • 8
  • Thanks, this is really helpful! I'm thinking to put it in a loop so that bottom line the daemon runs permanently. In case it stops I think about a solution as described here, that a cronjob controls if the script is running, if not an email or a restart ... : https://stackoverflow.com/questions/2366693/run-cron-job-only-if-it-isnt-already-running?rq=1 – tempo Nov 17 '19 at 19:07
  • The last `wait` seems useless. Add `echo "$(date): 6 jobs finished" >> logs/result.log` when you want to show that code will follow after the last `wait`. – Walter A Nov 18 '19 at 14:41
  • The last `wait`is not useless , when i run the master script , when it will finish , i am sure that all substaks was executed and finished – EchoMike444 Nov 18 '19 at 16:22
  • How is the "wait" command reacting if a script exits unexpectedly? Working hard to avoid this but sometimes it happens due to server load or due to reading of unexpected data format. Thanks in advance! – tempo Nov 24 '19 at 20:43
  • `wait` is waiting for completions of all subprocess ... completions can be unexpected – EchoMike444 Nov 24 '19 at 23:31