Why conditional statements are not processed rightly in PowerShell multithreading?

Question

I'm trying to get the dll and exe file names in the windows system directory (C:\Windows\System32).

There are too many files in that path, so I'm trying to use multithreading.

However, when comparing(at "IF" statement) extensions in ScriptBlock added by Addscript, only "False" result occurs, so the file name cannot be added into the list.

if ($extensionList -contains $tmpExtension.ToString())

Maybe the conditional statement is wrong or the ScriptBlock is wrong? The conditional statement works normally when multithreading is not used.

Entire code(Powershell 5.1) :

$extensionList = @(".DLL",".EXE")
$systemDir = "C:\Windows\System32\"

$threadNumber = 4 #threads
$tempList = @()

$ScriptBlock = {
    Param ($tmpFilePath)
    $tmpExtension = [IO.Path]::GetExtension($tmpFilePath)
    if ($extensionList -contains $tmpExtension.ToString()) {
        Return $tmpFilePath
    }
    Return $null
}
 
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, $threadNumber)
$RunspacePool.Open()
$Jobs = @()

$tmpfileNameList = (Get-ChildItem -Path $systemDir -File) | select-object Name

foreach($tmpfileName in $tmpfileNameList) {
    $tmpFilePath = $systemDir + $tmpfileName.Name
    $Job = [powershell]::Create().AddScript($ScriptBlock).AddArgument($tmpFilePath)
    $Job.RunspacePool = $RunspacePool
    $Jobs += New-Object PSObject -Property @{
      Pipe = $Job
      Result = $Job.BeginInvoke()
   }
}

while ($Jobs.IsCompleted -contains $false) {Start-Sleep -Milliseconds 100}
 
$Results = @()
ForEach ($Job in $Jobs){
    if ($Job.Pipe.EndInvoke($Job.Result) -ne $null) {
        $tempList += $Job.Pipe.EndInvoke($Job.Result)
    }
}
foreach($tmp in $tempList) {
    Write-Host $tmp
}

I don't multithread enough to take this one but ... if I just do `$Results = Get-ChildItem -Path 'C:\windows\System32' -File -Recurse | Where Extension -in '.dll','.exe'`, I get the result in just a second. If this is as disk speed issue, then multi-threading multiple operation on the files might not speed things up. — Sage Pourpre, Nov 26 '21 at 11:27
[`$null` should be on the left side of equality comparison](https://stackoverflow.com/q/58217913/1701026). Besides, if you are concerned with multithreading (performance) you should [avoid using the increase assignment operator (`+=`) to create a collection](https://stackoverflow.com/a/60708579/1701026) as it is exponentially expensive. — iRon, Nov 27 '21 at 08:51

Santiago Squarzon · Accepted Answer · 2021-11-26T14:45:55.377

The main issue is that your powershell instance has no idea what $extensionList is since it's never being passed as an argument hence it is always evaluating to:

$null -contains [IO.Path]::GetExtension($tmpFilePath)

If your intent is to return the FullName of the files where it's extension is .dll or .exe your scriptblock should be like this:

$ScriptBlock = {
    Param ($tmpFilePath, $extensionList)

    if ($extensionList -contains [IO.Path]::GetExtension($tmpFilePath))
    {
        $tmpFilePath
    }
}

And the foreach loop would look like this:

$tmpfileNameList = (Get-ChildItem -Path $systemDir -File).FullName
$params = @{
    extensionList = $extensionList
}

$jobs = foreach($path in $tmpfileNameList)
{
    $params.tmpFilePath = $path
    
    $Job = [powershell]::Create().AddScript($ScriptBlock).AddParameters($params)
    $Job.RunspacePool = $RunspacePool

    [pscustomobject]@{
        Pipe = $Job
        Result = $Job.BeginInvoke()
    }   
}

See PowerShell.AddParameters Method for details.

Then this line is wrong:

while ($Jobs.IsCompleted -contains $false) {
    Start-Sleep -Milliseconds 100
}

Should be changed too:

while ($Jobs.Result.IsCompleted -contains $false) {
    Start-Sleep -Milliseconds 100
}

Lastly, you can get the results of the runspaces like:

$results = $jobs.ForEach({ $_.Pipe.EndInvoke($_.Result) })

PS /> $results

In addition to whats mentioned above, the logic needs to be improved for the script to be actually efficient.

Instead of passing one path to each powershell instance, you should split the array of paths ($tmpfileNameList) into chunks and give each instance a chunk to process.

You can use something like this to split your array:

$groupSize = [math]::Ceiling($tmpfileNameList.Count / $threadNumber)
$counter = [pscustomobject]@{ Value = 0 }
$groups = $tmpfileNameList | Group-Object -Property {
    [math]::Floor($counter.Value++ / $groupSize)
}

In this case, since you're using 4 threads, $groups will contain 4 groups. If you want to apply this technique to your code you can copy the logic from this link.

happy to help @dinnershow – Santiago Squarzon Nov 29 '21 at 02:04 — Santiago Squarzon, Nov 29 '21 at 02:04

Why conditional statements are not processed rightly in PowerShell multithreading?

1 Answers1