PowerShell: Is there a way to get the total count of objects piped into a function through a pipeline?

Question

Edit: Adding clarification this my desire is only to show a progress bar with an known end (e.g. end of the pipeline) so that the function can provide a percentage toward completion (usually for large sets in the hundreds or thousands). I occasionally write functions to take objects from both pipeline or via a parameter so that the function can be flexible.

A Progress bar for an array of objects coming in from a parameter is simple enough, and is equivalent to pulling in the full pipeline set first, then processing them again. I have been avoiding the latter and I simply forego the write-progress in this scenario as it's not worth the impact.

I don't recall where I saw it, but someone mentioned $PSCmdlet.MyInvocation might provide the count, but perhaps I interpreted that incorrectly.

---

I've written some functions that take pipeline input and often would like to write a percentage progress bar for all the objects coming into the function via the pipeline.

Is there a way to get the total count at the beginning of the function?

I'm aware of how increase a counter as the function loops through the pipeline objects, but this only gets me the number of objects processed so far. I'd like to get a percentage of this by calculating it against the full pipeline count.

I've looked at the $MyInvocation and $PSCmdlet.MyInvocation properties, but the PipelineLength and PipelinePosition values area always '2' no matter how big the pipeline set is.

Note: this isn't what I'm focusing on as a solution, it's just one of the things I found that looked promising

Here's a test:

Function Test-Pipeline {
[CmdletBinding()]

PARAM(
    [Parameter(ValueFromPipeLine=$true,ValueFromPipelineByPropertyName=$true)]
    [Alias("FullName")]
    [psobject[]]$Path
)

BEGIN{
    $PSCmdlet.MyInvocation | Select *
}
PROCESS{
    ForEach ($xpath in $Path) {
        $filepath = Resolve-Path $xpath | Select -ExpandProperty Path
    }
}
END{}
}

When I pipe in the contents of dir (which this folder contains 20 items), I get:

PS C:\Temp> Dir | Test-Pipeline

MyCommand             : Test-Pipeline
BoundParameters       : {}
UnboundArguments      : {}
ScriptLineNumber      : 1
OffsetInLine          : 7
HistoryId             : 213
ScriptName            : 
Line                  : Dir | Test-Pipeline
PositionMessage       : At line:1 char:7
                        + Dir | Test-Pipeline
                        +       ~~~~~~~~~~~~~
PSScriptRoot          : 
PSCommandPath         : 
InvocationName        : Test-Pipeline
PipelineLength        : 2
PipelinePosition      : 2
ExpectingInput        : True
CommandOrigin         : Runspace
DisplayScriptPosition : 

Answer 1

As already commented, if you want to know the amount of objects you will receive in your cmdlet to predefine the units of a progress bar you basically can't (without breaking the streaming process).
Any "solution" that appears to do what you want, actually chokes the whole pipeline by collecting all the objects (in memory), counting them and finally releasing them all at once. This will violate the Strongly Encouraged Development Guidelines to Write Single Records to the Pipeline and result in a high memory usage.

Explanation

Visualize a assembly line, you are responsible of coloring all the objects that pass at your station. Now you want to know how many object you need to do today. Unless somebody outside your station (cmdlet) tells you how many will follow, you can only determine that by just receiving all the objects counting them, (still coloring them) and passing them on. As all these actions will take time (and storageroom for the objects), the person at the next station won't be happy as he will receive all the objects in once and a lot later than expected...

Technically

Let's build a cmdlet which generates colorless objects from a ProcessList:

function Create-Object {
    [CmdletBinding()]
    param(
        [Parameter(ValueFromPipeLine=$true)]$ProcessList,
        [Switch]$Show
    )
    begin {
        $Index = 0
    }
    process {
        $Object = [PSCustomObject]@{
            Index = $Index++
            Item  = $ProcessList
            Color = 'Colorless'
        }
        if ($Show) { Write-Host 'Created:' $Object.PSObject.Properties.Value }
        $Object
    }
}

And this is you, coloring the objects:

function Color-Object {
    [CmdletBinding()]
    param(
        [Parameter(ValueFromPipeLine=$true)]$InputObject,
        [Switch]$Show
    )
    process {
        $InputObject.Color = [ConsoleColor](Get-Random 16)
        if ($Show) { Write-Host 'Colored:' $InputObject.PSObject.Properties.Value }
        $InputObject
    }
}

This will be the result:

'a'..'e' |Create-Object |Color-Object

Index Item       Color
----- ----       -----
    0    a DarkMagenta
    1    b      Yellow
    2    c        Blue
    3    d        Gray
    4    e       Green

Now let's see, how things are actually processed:

'a'..'e' |Create-Object -Show |Color-Object -Show

Created: 0 a Colorless
Colored: 0 a DarkGreen

Created: 1 b Colorless
Colored: 1 b DarkRed
Created: 2 c Colorless
Colored: 2 c Gray
Created: 3 d Colorless
Colored: 3 d DarkGreen
Created: 4 e Colorless
Colored: 4 e DarkGray
Index Item     Color
----- ----     -----
    0    a DarkGreen
    1    b   DarkRed
    2    c      Gray
    3    d DarkGreen
    4    e  DarkGray

As you see, the first item "a" (index 0) is colored before the second item "b" (index 1) is created!
In other words, the Create-Object hasn't created all the objects yet and there is no way to know how many will follow. Except from just waiting for them which you do not want to do as explained before and certainly want to avoid if there are a lot of objects, the objects are fat (and a PowerShell object is usually fat) or in case of a slow input (see also: Advocating native PowerShell). This implies that you might want to make an exception to this if enumerating (counting) the objects is trivial to the rest of the process: e.g. collect file info (Get-ChildItem) to later invoke a heavy process (as e.g. Get-FileHash).

Answer 2

If you write the function without a Begin, Process and End block, you can use the $inputAutomatic variable to figure out how many items are sent through the pipeline.

$input contains an enumerator that enumerates all input that is passed to a function. The $input variable is available only to functions and script blocks (which are unnamed functions).

function Test-Pipeline {
    [CmdletBinding()]
    param(
        [Parameter(ValueFromPipeLine=$true,ValueFromPipelineByPropertyName=$true)]
        [Alias("FullName")]
        [psobject[]]$Path
    )
    $count = @($input).Count
    Write-Host "$count items are sent through the pipeline"

    # changed variable '$xpath' into '$item' to avoid confusion with XML navigation using XPath
    foreach ($item in $Path) {
        # process each item
        # $filepath = Resolve-Path $item | Select -ExpandProperty Path
    }
}

Get-ChildItem -Path 'D:\Downloads' | Test-Pipeline

Output something like

158 items are sent through the pipeline

Answer 3

You can't do that inside the BEGIN. The only way would be add the elements from the pipe in a list THEN run you code at the end.

Here's a way to do just that:

Function Test-Pipeline {
    [CmdletBinding()]
    
    PARAM(
        [Parameter(ValueFromPipeLine = $true, ValueFromPipelineByPropertyName = $true)]
        [Alias("FullName")]
        [psobject[]]$Path
    )
    
    BEGIN {
        $i = 0
        $list = @()
    }
    PROCESS {
        ForEach ($xpath in $Path) {
            $list += $xpath
            $i++
        }
    }
    END {
        $i
        if ($i -gt 0) {
            $j = 1
            ForEach ($item in $list) {
                Write-Output $item
                Write-Progress -Activity 'activity' -Status $item.ToString() -PercentComplete (($j++)*100/$i)
                Start-Sleep -Seconds 2
            }
        }
    }
}

Answer 4

If you need the count, you can use the Tee-Object Cmdlet to create a new var.

dir | tee-object -Variable toto | Test-Pipeline

Then

Function Test-Pipeline {
[CmdletBinding()]

PARAM(
    [Parameter(ValueFromPipeLine=$true,ValueFromPipelineByPropertyName=$true)]
    [Alias("FullName")]
    [psobject[]]$Path
)

BEGIN{
    $PSCmdlet.MyInvocation | Select *
    $toto.count

}
PROCESS{
    ForEach ($xpath in $Path) {
        $filepath = Resolve-Path $xpath | Select -ExpandProperty Path
    }
}
END{}
}

Gives the following for me :

MyCommand             : Test-Pipeline
BoundParameters       : {}
UnboundArguments      : {}
ScriptLineNumber      : 1
OffsetInLine          : 35
HistoryId             : 11
ScriptName            : 
Line                  : dir | tee-object -Variable toto | Test-Pipeline
PositionMessage       : Au caractère Ligne:1 : 35
                        + dir | tee-object -Variable toto | Test-Pipeline
                        +                                   ~~~~~~~~~~~~~
PSScriptRoot          : 
PSCommandPath         : 
InvocationName        : Test-Pipeline
PipelineLength        : 3
PipelinePosition      : 3
ExpectingInput        : True
CommandOrigin         : Runspace
DisplayScriptPosition : 

73

And

dir | Measure-Object


Count    : 73
Average  : 
Sum      : 
Maximum  : 
Minimum  : 
Property :