5

We wrote a script that supposed to execute Azure PowerShell commands in parallel. The problem is when we increase -ThrottleLimit higher than one, some of the commands are not being performed properly. The script is:

# Writing IPs for whitelisting into file.
Add-Content -Path IPs.txt -Value ((Get-AzWebApp -ResourceGroupName "ResourceGroup1" -Name "WebApp1").OutboundIpAddresses).Split(",")
Add-Content -Path IPs.txt -Value ((Get-AzWebApp -ResourceGroupName "ResourceGroup1" -Name "WebApp1").PossibleOutboundIpAddresses).Split(",")
# Writing new file with inique IPs.
Get-Content IPs.txt | Sort-Object -Unique | Set-Content UniqueIPs.txt

# Referencing the file.
$IPsForWhitelisting = Get-Content UniqueIPs.txt

# Assigning priotiry number to each IP
$Count = 100
$List = foreach ($IP in $IPsForWhitelisting) { 
  $IP|Select @{l='IP';e={$_}},@{l='Count';e={$Count}}
  $Count++
}
# Whitelisting all the IPs from the list.
$List | ForEach-Object -Parallel {
    $IP = $_.IP
    $Priority = $_.Count
    $azureApplicationId ="***"
    $azureTenantId= "***"
    $azureApplicationSecret = "***"
    $azureSecurePassword = ConvertTo-SecureString $azureApplicationSecret -AsPlainText -Force
    $credential = New-Object System.Management.Automation.PSCredential($azureApplicationId , $azureSecurePassword)
    Connect-AzAccount -Credential $credential -TenantId $azureTenantId -ServicePrincipal | Out-null
    echo "IP-$Priority"
    echo "$IP/24"
    echo $Priority
    Add-AzWebAppAccessRestrictionRule -ResourceGroupName "ResourceGroup1" -WebAppName "WebApp1" -Name "IP-$Priority" -Priority $Priority -Action Allow -IpAddress "$IP/24"
} -ThrottleLimit 1

If ThrottleLimit is set to 1 - 8 rules are being created, if ThrottleLimit is set to 2 - 7 rules are being created, 3 - 4 rules, 10 - 1 rule, hence some rules are being skipped.

What is the reason for such behavior?

WinBoss
  • 879
  • 1
  • 17
  • 40
  • When you specify `-ThrottleLimit 1` you are limiting the number of threads to 1, which is not truly multithreading. – Martin Meli Jul 05 '21 at 13:15
  • Yes, `-ThrottleLimit 1` is here to demonstrate that the script works, while `-ThrottleLimit 2` is skipping threads. – WinBoss Jul 05 '21 at 13:57

1 Answers1

0

In short - the -Parallel parameter does not (yet perhaps) magically import all dependent variables that fall in the scope of the For-EachObject block. In reality PWSH spans separate processes and only the array that is looped over will be implicitly passed, all other variables need explicit designations.

One should use the $using: directive (prefix) to denote which variables are to be imported (made visible) in the parallel code block.

Example:

$avar = [Int]10
$bvar = [Int]20
$list = @('here', 'it', 'eees')

$list | ForEach-Object -Parallel {
    Write-Output "(a, b) is here ($($using:avar), $($using:bvar))"
    Write-Output "(a, b) missing ($($avar), $($bvar))"
    Write-Output "Current element is $_"
}```

*thus - the described behavior is likely due to the fact that config. variables are not imported (at all) and thus the operations silently fail.*
Stelf
  • 11
  • 1