How to wait all batch files to finish before exiting?

Question

I have a main batch file than calls 4 other batch files so we can run in parallel.

Example:

Main.bat

    start call batch1.bat
    start call batch2.bat
    start call batch3.bat
    start call batch4.bat

    exit

I want the Main.bat to exit after all the batch1 to batch 4 has stopped executing. In this way, I can get the total run time of the batch file. Problem is Main.bat exits even before batch1 to batch4 finishes executing.

I tried to compute for %errorlevel% for each batch file, but it always return 0 even though the 4 .bat files are still running.

Hoping someone could help me!

Thank you! :)

Aacini · Accepted Answer · 2017-03-14T02:25:54.413

22

I think this is the simplest and most efficient way:

@echo off

echo %time%

(
    start call batch1.bat
    start call batch2.bat
    start call batch3.bat
    start call batch4.bat
) | set /P "="

echo %time%

In this method the waiting state in the main file is event driven, so it does not consume any CPU time!

EDIT: Some explanations added

The set /P command would terminate when anyone of the commands in the ( block ) outputs a line, but start commands don't show any line in this cmd.exe. This way, set /P keeps waiting for input until all processes started by start commands ends. At that point the pipe line associated to the ( block ) is closed, so the set /P Stdin is closed and set /P command is terminated by the OS.

edited Mar 14 '17 at 02:25

answered Nov 07 '15 at 19:29

Aacini

65,180
12
72
108

1

I did get this to work, but each `batch?.bat` script has to use `EXIT` and not `EXIT /B` at the end. Also, using `START /B` was also not possible. It seems to require creating a new window. – lit Nov 07 '15 at 20:09
@Liturgist: I am afraid I don't understand how your comments are related to the question... **`:/`** – Aacini Nov 07 '15 at 20:35
I had to use `EXIT` at the end of each `batch?.bat` script. – lit Nov 07 '15 at 21:46
@Liturgist: This "problem" is related to the (not posted) batch?.bat files, _not_ to my solution on the posted main batch file. You should post such comment under the question. – Aacini Nov 07 '15 at 22:48
This is essentially a code-only answer. Those are not generally useful. Unless you explain, what the issue with question's code is, how this addresses that issue, and how it works, there is no way for anyone to make an educated decision, on whether or not this solves their issue. Advice must come with rationale, so that you know when good advice turns bad. Lacking that, you are propagating [cargo cult programming](https://en.wikipedia.org/wiki/Cargo_cult_programming). – IInspectable Mar 05 '17 at 14:14
1

@IInspectable: This "code-only answer" just adds **11 characters** to the original code. A solution as short as this one is generally evident for the reader even with no explanations. However, I added an explanation for you (the one that complained after 14 months). To know "what the issue with question's code is" you should read the question. To know "how this addresses that issue": this answer solves the problem described in the question. To "make an educated decision on whether or not this solves their issue" you should look at the check-mark in the answer: if it is green, the answer is Yes – Aacini Mar 14 '17 at 05:26
@IInspectable: If you still have doubts about this point, just copy the 6-lines solution code and test it by yourself, as generally is done! Your involved comment suggest me that you don't understand the question and hence you will not understand this answer without an extensive background, even if the solution is comprised of just 11 characters... – Aacini Mar 14 '17 at 05:27
2

`set /P` runs in another instance of cmd that has its `StandardInput` set to a pipe. Each `start` in the block also spawns a cmd instance. But because `start` creates a new console (if /B isn't used), Windows replaces the pipe `StandardOutput` on each cmd instance in the block with a handle for the attached console's screen buffer. But they still inherited a handle for the pipe, so when `set /P` calls `ReadFile` on its end, there are still potential writers and the read blocks until all of the cmd instances with a handle for the other end have exited. – Eryk Sun Mar 14 '17 at 11:16
1

You can force the pipe to close, causing `set /P` to return, if you use Process Explorer to manually close the pipe handle (in the handle list the name is `\Device\NamedPipe` since it's an anonymous pipe) in each of the cmd shell instances that are running the batch files. – Eryk Sun Mar 14 '17 at 11:22
@eryksun: Your explanation is more technicall than the mine, but basically similar; however, there is a point that is not clear to me. The `start` command _first_ creates a new console and establish the redirections of the standard handles for the new process, and _then_ spawns the new `cmd.exe` instance with these redirected handles. In this scheme I don't understand how _"they still inherited a handle for the pipe"_. Can a program in the new `cmd.exe` instance write on its `\Device\NamedPipe`? Which handle number it should use for this? – Aacini Mar 14 '17 at 15:54
2

The parent just passes the `CREATE_NEW_CONSOLE` flag to `CreateProcess`. The child sees that it's flagged to create a new console, so the startup code in kernelbase spawns an instance of conhost.exe (via IOCTLs to `ConDrv`). The child was created with the inherited pipe as its `StandardOutput` (see this using a debugger set to break early at `ntdll!RtlUserThreadStart`), but attaching to the new console overwrites its existing standard handles. It still has the inherited handle for the pipe; however, without knowing the handle value (some multiple of 4, like 68), it can't do anything with it. – Eryk Sun Mar 14 '17 at 18:16
@eryksun: This is interesting! Perhaps [`GetFileType`](https://msdn.microsoft.com/en-us/library/aa364960(v=vs.85).aspx) function could be tested on a series of values until find the one that returns `FILE_TYPE_PIPE`. How do you know that the pipe handle is a multiple of 4? – Aacini Mar 14 '17 at 20:22
The last two bits of a handle value are used for flags on special handles such as RPC handles. For example, before Windows 8 console input and screen buffer handles were not actual File handles, so they were flagged by setting both of these bits, to yield handle values like 3, 7, and 11. By flagging them the system could route console operations to special functions that used the LPC protocol to talk to the attached console host process -- such as `CreateFile` => `OpenConsoleW` and `ReadFile` => `ReadConsole` (or bypassing straight to a private internal function). – Eryk Sun Mar 14 '17 at 23:48
2

Windows 8 and later uses the condrv.sys device driver instead of LPC. Now console handles are opened as virtual files such as `\Device\ConDrv\Input` and `\Device\ConDrv\Output`. The handles are normal handles for kernel File objects (multiple of 4), and the regular NT I/O functions are used such as `NtCreateFile`, `NtClose`, `NtReadFile`, `NtWriteFile`, and `NtDeviceIoControlFile`. The old-school DOS devices are implemented as `CON` => `Device\ConDrv\Console`, `CONIN$` => `\Device\ConDrv\CurrentIn`, and `CONOUT$` => `\Device\ConDrv\CurrentOut`. – Eryk Sun Mar 14 '17 at 23:55
Process IDs are also always a multiple of 4, because they're assigned out of a handle table in the system process. – Eryk Sun Mar 14 '17 at 23:57
Great answer! Unfortunately when using a `for` loop to start the processes, I get "The process tried to write to a nonexistent pipe" and no processes. :-S – Michel de Ruiter Nov 09 '22 at 08:53
@MicheldeRuiter Try: `( for %%f in (*.bat) do start "" /B cmd /C "%%f" > NUL ) | pause` and redirect any standard output to the screen using `> CON`, that is, don't use a standard `echo Message` but an `echo Message > CON` one. I just tested this... **;)** – Aacini Nov 09 '22 at 13:59
@Aacini I also tested this and it works for two subprocesses, but starts to break when having 3 or more .bat files... – Michel de Ruiter Nov 10 '22 at 10:14

Stephan · Answer 2 · 2015-11-07T16:59:13.493

11

give a unique title string to the new processes, then check if any processes with this string in the title are running:

start "+++batch+++" batch1.bat
start "+++batch+++" batch2.bat
start "+++batch+++" batch3.bat
start "+++batch+++" batch4.bat
:loop
  timeout /t 1 >nul
  tasklist /fi "windowtitle eq +++batch+++*" |find "cmd.exe" >nul && goto :loop
echo all tasks finished

(find is used, because tasklist does not return a helpful errorlevel)

edited Nov 07 '15 at 16:59

answered Nov 07 '15 at 16:39

Stephan

53,940
10
58
91

1

Best Solution! The accepted solution is less flexible; e.g. can't put echo statements between start commands and the added CALL might require you to modify the bat file exit values. This is perfect! – gunslingor Jun 06 '19 at 13:50
@gunslingor I would think you could put echo statements in Aacini's code as long as you redirect them to the console and not STDOUT. Something to test for sure. – Squashman Sep 14 '21 at 13:48
@Squashman ... yeah, I don't really recall this page or what I was really doing here 2 years ago, lol... but it seems a month after making this comment I provide a solution I liked better for my application. – gunslingor Sep 15 '21 at 14:56
Works nicely - had to adapt because I'm running parallel Python scripts started using the script.py name, so `find` has to look for py.exe not cmd.exe. Shame tasklist won't list the window title in its output. – DisappointedByUnaccountableMod Oct 12 '21 at 15:49
@balmy: `tasklist /v` does (but is considerably slower, that's why I used `/fi` to filter for window title). – Stephan Oct 12 '21 at 17:00

score 5 · Answer 3 · answered Nov 07 '15 at 18:36

5

Give this a try.

@echo off
echo %time%
start "" /wait cmd /c bat1.bat |start "" /wait cmd /c bat2.bat |start "" /wait cmd /c bat3.bat
echo %time%

pause

answered Nov 07 '15 at 18:36

Squashman

13,649
5
27
36

the idea was to run them parallel, so `/wait` is contraproductive. – Stephan Nov 07 '15 at 19:23
very interesting... `cmd` is a neverending source of surprises :) – Stephan Nov 07 '15 at 19:33
3

Yes, run in parallel because the pipes `|`, _not_ because the `start` commands! – Aacini Nov 07 '15 at 19:35

score 0 · Answer 4 · answered Nov 07 '15 at 15:59

You could have batch1..batchn4 create flag files when they finish running.

e.g echo y > flag1 in batch1.bat

Then in the main batch file check for the existence of the flag files before exiting. You would need some sort of sleep utility to do something like this at the end of the main batch file:

IF EXIST flag1 GOTO check2
sleep <for a short amount of time>
goto check1

:check2
IF EXIST flag2 GOTO check3
sleep <for a short amount of time>
goto check2

:check3
IF EXIST flag3 GOTO check4
sleep <for a short amount of time>
goto check3

:check4
IF EXIST flag4 GOTO xit
sleep <for a short amount of time>
goto check4

:xit

The downside of this technique is that your timing is going to be off a little because you're polling for the flag files instead of being event driven. This may or may not be a problem in your situation. Batch files are pretty limited in this way. You might be better off trying to do it in PowerShell or python or some other more capable scripting language.

Thanks for the reply! Is it possible for the batch1..batchn to return something to the main batch file? I'm thinking of using that as a checker, %errorlevel% won't work though — roxsap, Nov 07 '15 at 16:03
Not that I'm aware of. You're starting them in separate processes so you can't do it with environment variables. — JJF, Nov 07 '15 at 16:12

score 0 · Answer 5 · answered Jul 12 '19 at 15:07

No one mentioned this solution, which I find to be the best. Put this at the end of your script, modify your process name if needed and the search strings.

This includes modifications necessary to have this work remotely too which was a headache. Originally I used tasklist instead of WMIC and checked named window titles defined when START is called, but windowName is N/A in tasklist when using it remotely. The powershell command worked better than timeout in our circumstances, which required us to run from this from a Jenkins agent.

    :WAIT_FOR_FINISH
        ECHO Waiting...
        powershell -command "Start-Sleep -Milliseconds 5000" > nul
        WMIC PROCESS WHERE Name="cmd.exe" GET commandline | findstr "searchString1 searchString 2" > nul && goto :WAIT_FOR_FINISH
        ECHO.
        ECHO Processes are Complete!
        ECHO.

How to wait all batch files to finish before exiting?

5 Answers5

Linked

Related