I've made a very versitile function to do this for you, and unlike the other answers you dont need to rejig your code to get it to work.
Just pass your function as a parameter to Async and pipe your input in, each item on the pipeline will run your scriptblock in parallel asynchronously, and emit them as each one is completed.
For your question specifically it would look something like this
@(
@{vmxFilePath='a';machineName='b';username='c';password='d';scriptTpath='e';scriptFile='f';uacDismissScript='g';snapshotName'h'},
@{vmxFilePath='i';machineName='j';username='k';password='l';scriptTpath='m';scriptFile='n';uacDismissScript='o';snapshotName'p'}
...
) `
| Async `
-Func { Process {
Execute $_.vmxFilePath $_.machineName $_.username $_.password $_.scriptTpath $_.scriptFile $_.uacDismissScript $_.snapshotName
} }
On top of this my function supports not only automatic construction of [powershell] (@binarySalt's answer) but also Jobs (utilised in @Joost's, but dont use these as they are much slower than runspaces) and Tasks if you're using other people's code that already spawn them (use the -AsJob flag, which I explain at the bottom of this answer).
So, that's not useful for new visitors to this question, lets do something more demonstrable that you can run on your machine and see real-world results.
Take this simple code for example, it just takes in some test data for websites and checks if they're up.
$in=TestData | ?{ $_.proto -eq 'tcp' }
$in `
| %{
$WarningPreference='SilentlyContinue'
$_ `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Test-NetConnection `
-ComputerName $_.address `
-Port $_.port `
-InformationLevel Quiet
)
} `
| Timer -name 'normal' `
| Format-Table
Here's the test data, just a good few of the same websites on repeat.
And also a timing function to see how performant it is.
Function TestData {
1..20 | %{
[PsCustomObject]@{proto='tcp' ; address='www.w3.org' ; port=443},
[PsCustomObject]@{proto='https'; address='www.w3.org' ; port=443},
[PsCustomObject]@{proto='icmp' ; address='www.w3.org' ; },
[PsCustomObject]@{proto='tcp' ; address='developer.mozilla.org' ; port=443},
[PsCustomObject]@{proto='https'; address='developer.mozilla.org' ; port=443},
[PsCustomObject]@{proto='icmp' ; address='developer.mozilla.org' ; },
[PsCustomObject]@{proto='tcp' ; address='help.dottoro.com' ; port=80 },
[PsCustomObject]@{proto='http' ; address='help.dottoro.com' ; port=80 },
[PsCustomObject]@{proto='icmp' ; address='help.dottoro.com' ; }
}
}
Function Timer {
Param ($name)
Begin {
$timer=[system.diagnostics.stopwatch]::StartNew()
}
Process { $_ }
End {
@(
$name,
' '
[math]::Floor($timer.Elapsed.TotalMinutes),
':',
($timer.Elapsed.Seconds -replace '^(.)$','0$1')
) -join '' | Out-Host
}
}
Okay, 15seconds, so how much faster can this get if we use Async?
And how much do we have to change to get it to work?
$in=TestData | ?{ $_.proto -eq 'tcp' }
$in `
| Async `
-Expected $in.Count `
-Func { Process {
$WarningPreference='SilentlyContinue'
$_ `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Test-NetConnection `
-ComputerName $_.address `
-Port $_.port `
-InformationLevel Quiet
)
} } `
| Timer -name 'async' `
| Format-Table
It looks basically identical..
Okay, what's the speed?
Wow, cut it by two thirds!
Not only that, but because we know how many items are in the pipeline I wrote in some smarts to give you a progressbar and an ETA
Don't believe me? Have a video
Or run the code yourself :)
#Requires -Version 5.1
#asynchronously run a pool of tasks,
#and aggregate the results back into a synchronous output
#without waiting to pool all input before seeing the first result
Function Async { Param(
#maximum permitted simultaneous background tasks
[int]$BatchSize=[int]$env:NUMBER_OF_PROCESSORS * 3,
#the task that accepts input on a pipe to execute in the background
[scriptblock]$Func,
#because your task is in a subshell you wont have access to your outer scope,
#you may pass them in here
[array]$ArgumentList=@(),
[System.Collections.IDictionary]$Parameters=@{},
#the title of the progress bar
[string]$Name='Processing',
#your -Func may return a [Job] instead of being backgrounded itself,
#if so it must return @(job;input;args)
#optionally job may be a [scriptblock] to be backgrounded, or a [Task]
[switch]$AsJob,
#if you know the number of tasks ahead of time,
#providing it here will have the progress bar show an ETA
[int]$Expected,
#outputs of this stream will be @(job;input) where job is the result
[switch]$PassThru,
#the time it takes to give up on one job type if there are others waiting
[int]$Retry=5
)
Begin {
$ArgumentList=[Array]::AsReadOnly($ArgumentList)
$Parameters=$Parameters.GetEnumerator() `
| &{
Begin { $params=[ordered]@{} }
Process { $params.Add($_.Key, $_.Value) }
End { $params.AsReadOnly() }
}
#the currently running background tasks
$running=@{}
$counts=[PSCustomObject]@{
completed=0;
jobs=0;
tasks=0;
results=0;
}
#a lazy attempt at uniquely IDing this instance for Write-Progress
$asyncId=Get-Random
#a timer for Write-Progress
$timer=[system.diagnostics.stopwatch]::StartNew()
$pool=[RunspaceFactory]::CreateRunspacePool(1, $BatchSize)
$pool.Open()
#called whenever we want to update the progress bar
Function Progress { Param($Reason)
#calculate ETA if applicable
$eta=-1
$total=[math]::Max(1, $counts.completed + $running.Count)
if ($Expected) {
$total=[math]::Max($total, $Expected)
if ($counts.completed) {
$eta=`
($total - $counts.completed) * `
$timer.Elapsed.TotalSeconds / `
$counts.completed
}
}
$Reason=Switch -regex ($Reason) {
'^done$' { "Finishing up the final $($running.Count) jobs." }
'^(do|next)$' { "
Running
$($running.Count)
jobs concurrently.
$(@('Adding','Waiting to add')[!($Reason -eq 'do')])
job #
$($counts.completed + $running.Count + 1)
" -replace '\r?\n\t*','' }
Default { "
Running $($running.Count) jobs concurrently.
Emitting
$($counts.completed)
$(@{1='st';2='nd';3='rd'}[$counts.completed % 10] -replace '^$','th')
result.
" -replace '\r?\n\t*','' }
}
Write-Progress `
-Id $asyncId `
-Activity $Name `
-SecondsRemaining $eta `
-Status ("
$($counts.completed)
jobs completed in
$([math]::Floor($timer.Elapsed.TotalMinutes))
:
$($timer.Elapsed.Seconds -replace '^(.)$','0$1')
" -replace '\r?\n\t*','') `
-CurrentOperation $Reason `
-PercentComplete (100 * $counts.completed / $total)
}
#called with the [Job]'s that have completed
Filter Done {
++$counts.completed
$out=$running.Item($_.Id)
$running.Remove($_.Id)
Progress
$out.job=`
if ($_ -is [System.Management.Automation.Job]) {
--$counts.jobs
$_ | Receive-Job
}
elseif ($_.pwsh) {
--$counts.results
try {
$_.pwsh.EndInvoke($_)
}
catch {
#[System.Management.Automation.MethodInvocationException]
$_.Exception.InnerException
}
finally {
$_.pwsh.Dispose()
}
}
elseif ($_.IsFaulted) {
--$counts.tasks
#[System.AggregateException]
$_.Exception.InnerException
}
else {
--$counts.tasks
$_.Result
}
if ($PassThru) {
$out
}
else {
$out.job
}
}
$isJob={
$_ -is [System.Management.Automation.Job]
}
$isTask={
$_ -is [System.Threading.Tasks.Task]
}
$isResult={
$_ -is [IAsyncResult]
}
$isFinished={
$_.IsCompleted -or `
(
$_.JobStateInfo.State -gt 1 -and
$_.JobStateInfo.State -ne 6 -and
$_.JobStateInfo.State -ne 8
)
}
$handle={
$_.AsyncWaitHandle
}
Function Jobs { Param($Filter)
$running.Values | %{ $_.job } | ? $Filter
}
#called whenever we need to wait for at least one task to completed
#outputs the completed tasks
Function Wait { Param([switch]$Finishing)
#if we are at the max background tasks this instant
while ($running.Count -ge $BatchSize) {
Progress -Reason @('done','next')[!$Finishing]
$value=@('jobs', 'tasks', 'results') `
| %{ $counts.($_) } `
| measure -Maximum -Sum
$wait=if ($value.Maximum -lt $value.Sum) {
$Retry
}
else {
-1
}
$value=Switch -exact ($value.Maximum) {
$counts.jobs {
(Wait-Job `
-Any `
-Job (Jobs -Filter $isJob) `
-Timeout $wait
).Count -lt 1
break
}
Default {
[System.Threading.WaitHandle]::WaitAny(
(Jobs -Filter $handle | % $handle),
[math]::Max($wait * 1000, -1)
) -eq [System.Threading.WaitHandle]::WaitTimeout
break
}
}
(Jobs -Filter $isFinished) | Done
}
}
}
#accepts inputs to spawn a new background task with
Process {
Wait
Progress -Reason 'do'
$run=[PSCustomObject]@{
input=$_;
job=$Func;
args=$ArgumentList;
params=$Parameters;
}
if ($AsJob) {
$run.job=$NULL
Invoke-Command `
-ScriptBlock $Func `
-ArgumentList @($run) `
| Out-Null
}
if ($run.job | % $isJob) {
++$counts.jobs
}
elseif ($run.job | % $isTask) {
++$counts.tasks
}
#if we weren't given a [Job] we need to spawn it for them
elseif ($run.job -is [ScriptBlock]) {
$pwsh=[powershell]::Create().AddScript($run.job)
$run.args | %{ $pwsh.AddArgument($_) } | Out-Null
$pwsh.RunspacePool=$pool
$run.job=$pwsh.AddParameters($run.params).BeginInvoke(
[System.Management.Automation.PSDataCollection[PSObject]]::new(
[PSObject[]]($run.input)
)
)
$run.job | Add-Member `
-MemberType NoteProperty `
-Name pwsh `
-Value $pwsh `
-PassThru `
| Add-Member `
-MemberType NoteProperty `
-Name Id `
-Value $run.job.AsyncWaitHandle.Handle.ToString()
++$counts.results
}
else {
throw "$($run.job.GetType()) needs to be a ScriptBlock"
}
$running.Add($run.job.Id, $run) | Out-Null
}
End {
#wait for the remaining running processes
$BatchSize=1
Wait -Finishing
Write-Progress -Id $asyncId -Activity $Name -Completed
$pool.Close()
$pool.Dispose()
}
}
So you may have noticed three things above, I alluded to -AsJob (using a mix of Jobs, Tasks and scriptblocks), there are unused protocols mentioned in the test data, and the video had a third test in it.
Here it is. Instead of doing the basic tcp test, using the test data we will also do a http/s check and a icmp ping (idk, maybe the https fails, but you want to narrow down if it's because the machine is down or just the service).
$in=TestData
$in `
| Async `
-Expected $in.Count `
-PassThru `
-AsJob `
<#this would be accessible as a named parameter if needed#>`
-Parameters @{proxy=[System.Net.WebRequest]::GetSystemWebProxy()} `
-Func { Param([parameter(Position=0)]$x)
$x.job=Switch -regex ($x.input.proto) {
'^icmp$' {
Test-Connection `
-ComputerName $x.input.address `
-Count 1 `
-ThrottleLimit 1 `
-AsJob
}
'^tcp$' {
$x.params=@{address=$x.input.address; port=$x.input.port}
{ Param($address, $port)
$WarningPreference='SilentlyContinue'
Test-NetConnection `
-ComputerName $address `
-Port $port `
-InformationLevel Quiet
}
}
'^(http|https)$' {
[Net.ServicePointManager]::SecurityProtocol=[Net.SecurityProtocolType]::Tls12
$request=[System.Net.HttpWebRequest]::Create((@(
$x.input.proto,
'://',
$x.input.address,
':',
$x.input.port
) -join ''))
$request.Proxy=$NULL
$request.Method='Get'
$request.GetResponseAsync()
}
}
} `
| %{
$result=$_
$result.input `
| Add-Member `
-PassThru `
-MemberType NoteProperty `
-Name result `
-Value $(Switch -regex (@($result.input.proto, $result.job.message)[$result.job -is [Exception]]) {
#[Win32_PingStatus]
'^icmp$' { $result.job.StatusCode -eq 0 }
#[bool]
'^tcp$' { $result.job }
#[System.Net.HttpWebResponse]
'^(http|https)$' {
$result.job.Close()
Switch ($result.job.StatusCode.value__) {
{ $_ -ge 200 -and $_ -lt 400 } { $True }
Default {$False}
}
}
#[Exception]
Default { $False }
})
} `
| Timer -name 'async asjob' `
| Format-Table
As you may have seen, this did more than double the work of the original code, but still finished in around half the time at 8seconds.
