Runtime of Foreach-Object vs Foreach loop

Question

I want to do a progress bar of my script but then I need a total amount of folders.

Is there a significant runtime difference between:

Get-ChildItem $path -Directory | ForEach-Object {
    #do work
}

and

$folders = Get-ChildItem $path -Directory
foreach($folder in $folders){
    #do work
}

Then I can use $folders.Count as my total amount of folders. I don't know how to do it with a foreach-object loop.

`I don't know how to do it with a foreach-object loop.` Not possible by design. — wOxxOm, Sep 22 '16 at 11:07
@wOxxOm Sure it's possible. You just need to increment a counter inside the `ForEach-Object`: `$i=0; ... | ForEach-Object { $i++; ... }; $i` — Ansgar Wiechers, Sep 22 '16 at 11:39
@AnsgarWiechers, it's meaningless in the discussed context. You're answering a technical aspect but lose sight of the overall goal which is to know number of folders before enumeration occurs. — wOxxOm, Sep 22 '16 at 12:05
@wOxxOm So you'd imagine that accumulating the entire list and the enumerate would be slower than enumerating as they appear right? But that is not the case? The other answers seems to say otherwise. — Benjamin S. Sorterup, Sep 23 '16 at 09:48
No. Accumulating the entire list first would introduce the initial delay which can be extremely big if you enumerate a big folder. — wOxxOm, Sep 23 '16 at 09:52

Avshalom · Answer 1 · 2023-03-21T08:59:40.997

7

You can check for yourself:

Measure-Command {
    1..100000 | ForEach-Object $_
}

# 1.17s

Measure-Command {
    foreach ($i in 1..100000)
    {
    $i
    }
}

# 0.15s

edited Mar 21 '23 at 08:59

answered Sep 22 '16 at 10:33

Avshalom

8,657
1
25
43

But you're excluding the Get-ChildItem part. First I will have to acquire all the folders and then loop instead of just looping through the folders. – Benjamin S. Sorterup Sep 22 '16 at 10:59
2

it doesn't matter, the point is to show the speed compare between the two options, anyway you can do `$Dirs = Get-ChildItem $path -Directory` then `$Dirs | Foreach-Object or foreach ($d in $Dirs) {}` – Avshalom Sep 22 '16 at 11:25
The [`Foreach-Object`](https://learn.microsoft.com/powershell/module/microsoft.powershell.core/foreach-object) cmdlet has indeed a little more overhead than the [foreach](https://learn.microsoft.com/powershell/module/microsoft.powershell.core/about/about_foreach) statement but you should not blindly break your pipeline for this (which means that you should understand the background of the [PowerShell Pipepline](https://learn.microsoft.com/powershell/module/microsoft.powershell.core/about/about_pipelines)) see: [Slow Execution of Powershell ForEach](https://stackoverflow.com/a/75798807/1701026) – iRon Mar 21 '23 at 08:46

score 7 · Accepted Answer · answered Sep 23 '16 at 10:01

Piping is designed to process items immediately as they appear so the entire length of the list is not known while it's being piped.

Get-ChildItem $path -Directory | ForEach {
    # PROCESSING STARTS IMMEDIATELY
    # LENGTH IS NOT KNOWN
}

Advantage: processing starts immediately, no delay to build the list.
Disadvantage: the list length is not known until it's fully processed

On the other hand, assigning the list to a variable builds the entire list at this point, which can take an extremely large amount of time if the list contains lots of items or it's slow to build, for example, if it's a directory with lots of nested subdirectories, or a slow network directory.

# BUILD THE ENTIRE LIST AND ASSIGN IT TO A VARIABLE
$folders = Get-ChildItem $path -Directory
# A FEW MOMENTS/SECONDS/MINUTES/HOURS LATER WE CAN PROCESS IT
ForEach ($folder in $folders) {
    # LENGTH IS KNOWN: $folders.count
}

Advantage of building the list + ForEach statement: overall time spent is less because processing { } block is not invoked on each item whereas with piping it is invoked like a function or scriptblock, and this invocation overhead is very big in PowerShell.
Disadvantage: the initial delay in the list assignment statement can be extremely huge

This may be the difference between enumerators and arrays. Enumerators are delayed computation, which in many cases will be more efficient and take up fewer resources. — Andy, May 01 '21 at 23:58

score 6 · Answer 3 · answered Sep 22 '16 at 09:36

Yes, there is a performance difference. foreach is faster than ForEach-Object, but requires more memory, because all items ($folders) must be in memory. ForEach-Object processes one item at a time as they're passed through the pipeline, so it has a smaller memory footprint, but isn't as fast as foreach.

Runtime of Foreach-Object vs Foreach loop

3 Answers3

Linked

Related