Improving Batch File for loop with start subcommand

Question

I've currently got a very simple script which pings 10 IPs:

@ECHO OFF
for /L %%i in (100, 1, 110) DO (
  START /W /B cmd /c ping -n 1 192.168.0.%%i | find "time="
)

The output is as expected:

C:\Users\herpderp>test.bat
Reply from 192.168.0.101: bytes=32 time=294ms TTL=64
Reply from 192.168.0.104: bytes=32 time=1ms TTL=64

However, it is VERY slow and definitely happening sequentially. When I run this with a PowerShell Measure-Command I get these results:

PS C:\Users\derpherp> measure-command {start-process test.bat -Wait}

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 23
Milliseconds      : 107
Ticks             : 231074173
TotalDays         : 0.000267446959490741
TotalHours        : 0.00641872702777778
TotalMinutes      : 0.385123621666667
TotalSeconds      : 23.1074173
TotalMilliseconds : 23107.4173

So we see it is taking ~23 seconds to execute.

Now, if I were on a Linux system and wanted to do this same thing, I can do the following:

#!/bin/bash

for ip in $(seq 100 110); do
  ping -c 1 192.168.0.$ip | grep "bytes from" | cut -d" " -f4 | cut -d ":" -f1 &
done

The result is different in a variety of ways:

• The results aren't always sequential, meaning it actually started the commands more asynchronously:

  root@kali:~/vids/bash_scripting# ./test.sh
  192.168.0.104
  192.168.0.100
  192.168.0.103
  192.168.0.101

• The time for it to display all of the positive results is typically less than a second and this scales to much larger sets of numbers.

So, my question, is this a limitation of batch scripting? I find it hard to believe that bash performs literally 100s of times better than Windows CMD interpreter for such a simple task?

If this is limited, does PowerShell offer a competitive way to create tasks? I've used Start-Job in a foreach loop but that seems to become unworkable for large numbers of tasks (ie Test-Connection on a /16 network)

Edit 1

@ECHO OFF
for /L %%i in (100, 1, 110) DO (
  ping -n 1 192.168.0.%%i | find "time="
)

This outputs the current window and takes just as long as the initial variant

@ECHO OFF
for /L %%i in (100, 1, 110) DO (
  START ping -n 1 192.168.0.%%i | find "time="
)

This outputs to unique windows per IP and takes just as long to complete.

Answer 1

Asynchronous isn't easy, but with jobs you can get close pretty easily with PowerShell.

This code should behave how it looks like your bash code does, returning the IP address of all hosts that respond without error:

$IPAddresses = 100..110 | ForEach-Object { "192.168.0.$_"; }

$JobList = $IPAddresses | ForEach-Object {
    Test-Connection -ComputerName $_ -Count 1 -AsJob;
}

Receive-Job -Job $JobList -AutoRemoveJob -Wait | `
    Where-Object { $_.StatusCode -eq 0 } | `
    Select-Object -ExpandProperty Address;

The above completes for me against ~250 hosts in 3 seconds.

Test-Connection itself claims to use up to 32 simultaneous connections (configurable with -ThrottleLimit setting) but it sure seems like the -Delay option completely overrides that setting if you're targeting a wide range.

You may have issues with PowerShell v4 and earlier, as it may behave slightly differently. Test-Connection in particular seems to be idiosyncratic on different versions of Windows or PowerShell. At least, I always remember it throwing an error instead of returning an error status code in previous versions.

If you want more fine grain control over the ping than Test-Connection gives you -- for example, it defaults to a 1 second timeout so the minimum time you'll wait when any host is down is 1 second -- you'll probably have to revert to directly calling Get-WMIObject -Query "SELECT * FROM Win32_PingStatus WHERE Address = '$IP' AND TimeOut = 200" -AsJob.

Answer 2

Seems that I have found a pure batch-file solution. Basically, the script fires several ping commands simultaneously, which all write their result to individual log files; these files are monitored for write-access, because the files are write-locked as long as the respective ping processes are ongoing; as soon as write-access is granted, the first lines of the log files containing the IP addresses are copied into a summary log file. So here is the code -- see all the explanatory rem remarks:

@echo off
setlocal EnableExtensions DisableDelayedExpansion

rem // Predefine IP addresses as an array:
for /L %%J in (0,1,5) do (
    set "IP[%%J]=127.0.0.%%J"
)

rem // Ping IP addresses simultaneously, create individual log files:
for /F "tokens=2,* delims=[=]" %%J in ('set IP[') do (
    start "" /B cmd /C ping -n 2 %%K ^| find "TTL=" ^> "%~dpn0_%%J.log"
)

rem // Deplete summary log file:
> "%~dpn0.log" rem/

rem /* Polling loop to check whether individual log files are accessible;
rem    this works, because a pinging process is not yet finished, the
rem    respective log file is still opened and therefore write-locked: */
:POLL
rem // Give processor some time:
> nul timeout /T 1 /NOBREAK
rem /* Loop through all available array elements; for every accomplished
rem    pinging process, the related array element becomes deleted, so
rem    finally, there should not be any more array elements defined: */
for /F "tokens=2 delims=[=]" %%J in ('2^> nul set IP[') do (
    rem // Suppress error message in case log file is write-locked:
    2> nul (
        rem // Try to append nothing to the log file:
        >> "%~dpn0_%%J.log" rem/ && (
            rem /* Appending succeeded, hence log file is no longer locked
            rem    and the respective pinging process has been finished;
            rem    therefore read the first line containing the IP: */
            set "FILE=" & set "IP="
            < "%~dpn0_%%J.log" set /P IP=""
            rem // Copy the read line to the summary log file:
            if defined IP >> "%~dpn0.log" call echo(%%IP%%
            rem // Undefine related array element:
            set "IP[%%J]="
            rem // Store log file path for later deletion:
            set "FILE=%~dpn0_%%J.log"
        )
        rem // Delete individual log file finally:
        if defined FILE call del "%%FILE%%"
    )
)
rem // Jump to polling loop in case there are still array elements:
> nul 2>&1 set IP[ && goto :POLL

endlocal
exit /B

Answer 3

You have no need of start command. You turn off all it's features. Then you tell it to start a new process for CMD which is slow and unnecessary. So - ping -n 1 192.168.0.%%i |find "time=".

See my summary here of how to start programs - What do all commands in batch mean and do?.

You can use start to run ping in parallel, but you need to remove /w which means wait (but it's being ignored because) and /b which says don't run the command in a new spawned process but do it sequentially.

As you are still doing multiple process creation of Ping you can use WMIC to avoid that and test all computers in one process create.

wmic /node:@"c:\computerlist.txt" /append:"textfile.txt" path win32_pingstatus where "address='127.0.0.1' and responsetime > 100" get responsetime,timestamprecord.

So we've gone from 20 process creates to 1.

In programming we have to pay taxes of system overhead. Windows collect most tax at Process Creation and Window Creation. This is to allow other operations to take place without as much system overheads. An example is Windows keeps tables of processes that are running - uses resources to update when a process is created but allows quick lookup for other operations.