9

How do I execute some tasks in parallel in batch script and wait for them?

command1;

# command3, command4 and command5 should execute in sequence say task1
# command6, command7 and command8 should execute in sequence say task2
# both task1 and task2 should run independently

command3; command4; command5 | command6; command7; command8;

# should execute only after the above parallel tasks are completed

command9;

As a proof of concept I tried something like but it is not working:

echo "Starting"
start /wait wait20.bat   
start /wait wait40.bat 
echo "Finishing"

wait20.bat looks like:

echo "starting 20 seconds job"
timeout 20
echo "finishing 20 seconds job"

What am I doing wrong?

aschipfl
  • 33,626
  • 12
  • 54
  • 99
Nishant
  • 20,354
  • 18
  • 69
  • 101
  • 1
    [possible duplicate](https://stackoverflow.com/questions/47657254/in-windows-batch-loop-how-to-wait-for-spawned-processes-to-complete-before-cont/47657549#47657549) – Stephan Mar 01 '18 at 13:28
  • Possible duplicate of [Waiting for parallel batch scripts](https://stackoverflow.com/questions/12577442/waiting-for-parallel-batch-scripts) – phuclv Mar 01 '18 at 14:27

2 Answers2

15

I think this is the simplest way to do that:

command1

(
   start "task1" cmd /C "command3 & command4 & command5"
   start "task2" cmd /C "command6 & command7 & command8"
) | pause

command9

Further details in the comments below this answer.

Aacini
  • 65,180
  • 12
  • 72
  • 108
  • This is so elegant! However, I have a doubt here. if I just do a `start task1`, it does it in the background without monitoring it. So what part of this code actually does the important part of waiting for the background tasks to complete? Is it the `()`, `|` or `pause`? I read the explanation in the comment but it talks about `set /P`. – Nishant Mar 01 '18 at 14:55
  • 1
    Is the pipe `|` combined with any command that read an input from stdin, like `pause` or `set /P`. Such a command will wait until the `start` command send an output (that will never happen) _or_ the background process ends... – Aacini Mar 01 '18 at 21:41
  • Also, if you are interleaving `timeout` commands with the `start` ones (to stagger the parallel execution), make sure to add `>nul` at the end of each `timeout` statement otherwise the `pause` will not work. – Burhan Mar 31 '22 at 22:50
  • @Burhan: I don't see how a `timeout` command could be useful in this case... Of course, if you insert _any command_ that output anything to the console, it would break the `pause` (not just the `timeout` command). – Aacini Apr 01 '22 at 19:01
  • @Aacini I had a bunch of build processes that I wanted to run in parallel, but they shared one key file. If all the processes started at the exact same time, only one of them would get a lock on the key file and rest would fail. However, staggering them by a second using `timeout` allowed enough time for every process to read the key file and release the lock before the next one would try locking it again. – Burhan Apr 06 '22 at 17:01
  • @Aacini Also, thanks for the clarification on any output breaking `pause`, I suspected as much but wasn't 100% sure since I only tried `timeout`. Maybe you can add that to your answer. – Burhan Apr 06 '22 at 17:05
1

Here's an example how it can be done with tasklist and process window titles:

launcher.cmd

@echo off
for /l %%a in (1,1,5) do start "worker%%a" cmd /c worker.cmd & timeout /t 1 >nul
:loop
timeout /t 2 >nul
tasklist /v /fi "imagename eq cmd.exe" /fo csv | findstr /i "worker" >nul && goto :loop
echo Workers finished

worker.cmd

@echo off
set /a wait=2 + ( %RANDOM% %% 5 ) 
echo waiting for %wait%...
timeout /t %wait% >nul
echo I'm done!
timeout /t 2 >nul

I'm using tasklist /v /fo csv to get titles.

wolfrevokcats
  • 2,100
  • 1
  • 12
  • 12
  • Thanks. Is there a way to not display `tasklist` and `timeout` commands in the console? – Nishant Mar 01 '18 at 14:18
  • Just redirect output to nul: `timeout /t NN >nul`, `findstr /i ... >nul` – wolfrevokcats Mar 01 '18 at 14:25
  • That still shows the `timeout` command, just not the **output**! – Nishant Mar 01 '18 at 14:31
  • Strange... I've just checked, and it is showing nothing for me. I've updated my post with `> nul` – wolfrevokcats Mar 01 '18 at 14:44
  • 1
    This method waste CPU time repeatedly executing `tasklist` and `findstr` commands until parallel processes ends. An [event driven method](https://stackoverflow.com/a/49051876/778560) is more efficient and simpler... – Aacini Mar 01 '18 at 14:44
  • @Aacini, how much CPU does it waste running every two seconds? – wolfrevokcats Mar 01 '18 at 14:47
  • 1
    @wolfrevokcats, the point Aacini is making is that it that the other approach is event-driven. It just wakes up the process when it is ready which is always cleaner than doing a poll. – Nishant Mar 01 '18 at 16:12
  • The problem I see with this approach is that the main batch waits indefinitely for others to finish, while polling approach allows you to check for some timeout. Otherwise, it is a nice trick to have. – wolfrevokcats Mar 01 '18 at 17:06