Array causing 'system.outofmemoryexception'

Question

I am running the below script and it is causing exception of type 'system.outofmemoryexception' was thrown

I believe that it is due to the @Results array growing past the 2gb allocated to windows shell. Is there possibly a way to iterate through the results, or am I stuck with allocating more memory (which could ultimately be a lot...)?

$Path = "path to output" 

### Get all PF
$publicFolders = Get-PublicFolder "Public Folder Name" -Recurse -resultsize unlimited | Select-Object *


### Array to contain results
$results = @()

###Begin looping through each PF and grab each user/group with AccessRights to that folder
$final = ForEach($pf in $publicFolders){    
    $perms     = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"} 
    
    Foreach($perm in $perms){
        $temp = [PSCustomObject]@{
                    MailFolderName = $pf.Identity 
                    UserWithPerms  = $perm.User
                    AccessRights   = $perm | Select-Object -ExpandProperty AccessRights

                }
                $results += $temp
    }
                   
}

$final | Export-Csv $path -NoTypeInformation

Am I barking up the wrong tree?

Thanks in advance.

Mathias' solution is preferable, but what likely contributed to your problem: Incrementally extending arrays in a loop with `+=` is inefficient, because a _new_ array must be created behind the scenes _in every iteration_, because arrays are of fixed size; a much more efficient approach is to use a `foreach` loop as an _expression_ and let PowerShell itself collect the outputs in an array: `[array] $outputs = foreach (...) { ... }` - see [this answer](https://stackoverflow.com/a/60708579/45375). — mklement0, Oct 08 '21 at 16:17

score 3 · Accepted Answer · answered Oct 08 '21 at 16:10

Use the ForEach-Object cmdlet instead of a foreach(){} loop statement for the outer loop - this way you can start piping the output to Export-Csv immediately instead of buffering it in an array:

$publicFolders |ForEach-Object {
    $pf    = $_
    $perms = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"} 
    
    Foreach($perm in $perms){
        [PSCustomObject]@{
            MailFolderName = $pf.Identity 
            UserWithPerms  = $perm.User
            AccessRights   = $perm | Select-Object -ExpandProperty AccessRights
        }
    }                   
} | Export-Csv $path -NoTypeInformation

Alternatively, flush the partial set of results to file after enumerating the permissions for each folder:

ForEach($pf in $publicFolders){    
    $perms     = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"} 
    
    $results = Foreach($perm in $perms){
        [PSCustomObject]@{
            MailFolderName = $pf.Identity 
            UserWithPerms  = $perm.User
            AccessRights   = $perm | Select-Object -ExpandProperty AccessRights
        }
    }

    # flush the buffer before moving to the next set of permissions
    $results |Export-Csv $path -NoTypeInformation -Append
}

Outstanding! So simple. Obviously, it worked. I need to get down into the weeds re: forEach vs forEach-object (among other things), but you Sir are genius. Thanks mate. As you slide down the bannister of life, may all the splinters point in the right direction. — Techknow, Oct 08 '21 at 16:54
@MarcEverlove Most excellent to hear, and you're very welcome! :) The short of it is that `foreach(){...}` is generally "low-overhead" - less memory and usually slightly faster processing time - but it doesn't support writing output to the pipeline. `ForEach-Object` _does_ write to the pipeline, but it doesn't handle flow control statements (`break`, `continue`) very well, incurs higher memory cost and takes an O(N) perf hit from input pipeline binding. `ForEach-Object` also does a bunch of things `foreach(){}` doesn't, but that's part of the longer version of it :) — Mathias R. Jessen, Oct 08 '21 at 17:16

Array causing 'system.outofmemoryexception'

1 Answers1

Linked