How to get all piped in objects at once when using Begin/Process/End blocks?

Question

I am stuck on a learning project I have set out on, the objective here was meant for me to learn this whole the Begin, Process and End blocks concept

My project consists of a function, that will let the user open a single Firefox window with a specific profile and optionally open one or more web pages via .url files saved on the C: drive. This is what I have so far:

Function Ask-Web{
    [CmdletBinding()]
    Param(
        [Parameter(Position = 1)]
        [ArgumentCompletions('andy','andrew','halley', "hanna")]
        [Array]$FirefoxProfile = 'Andy',
        [Parameter(ValueFromPipeline, Position = 2)]
        [ArgumentCompletions('DIYhome','DIYreddit','DIYexchange','DIYall', 'SelectWithFZF')]
        [Array]$Pages
    )
Begin{
    $FavouritePages = [ordered]@{
    DIYhome     = "https://www.diyUK.com"
    DIYreddit   = "https://www.reddit.com/r/DIY/submit"
    DIYexchange = "https://diy.stackexchange.com/questions/ask"
    # DIYall  Will handle this later
    } 
    
    [array]$Pages = If($Pages -eq "SelectWithFZF"){Get-ChildItem -Path "C:\temp\Ask Pages\" -Filter *.url*| FZF}Else{$FavouritePages."$Pages"}  ;the FZF utility will store all selected items into $Pages variable
    }
End{. 'C:\Program Files\Mozilla Firefox\firefox.exe' -p $FirefoxProfile $Pages}

}

The function is supposed to let the user select pages via argument completions, pre provide a web page from the pipeline or make a selection from folder C:\temp\Ask Pages\ using the utility FZF.

In all instances, the user is expected to provide one or more pages/urls:

Ask-Web -FirefoxProfile andy -Pages DIYhome,DIYreddit,DIYexchange    ; Open a singe Firefox window, with profile "Andy" and the pages DIYhome,DIYreddit and DIYexchange
Ask-Web -FirefoxProfile hanna -Pages DIYhome                         ; Open a singe Firefox window, with profile "Hanna" and the page DIYhome
Ask-Web -FirefoxProfile andrew -Pages SelectWithFZF                  ; Open a singe Firefox window, with profile "Andrew" and all the pages that were selected with FZF
Get-ChildItem .\Diy -filter *.url* | Ask-Web -FirefoxProfile hanna   ; Open a singe Firefox window, with profile "Hanna" and with all the pages provided by the pipeline

Where I seem to be stuck is getting PowerShell to give me all the pipeline input at once, rather than one at a time or the last pipeline item, which is what I get with the above function.

If I go with Process{. 'C:\Program Files\Mozilla Firefox\firefox.exe' -p $FirefoxProfile $Pages} then PowerShell will open a Firefox window for every item there is in $Pages, yet with End{. 'C:\Program Files\Mozilla Firefox\firefox.exe' -p $FirefoxProfile $Pages} only the last item in $Pages gets opened.

I tried:

Function Ask-Web{
    [CmdletBinding()]
    Param(
        [Parameter(Position = 1)]
        [ArgumentCompletions('andy','andrew','halley', "hanna")]
        [Array]$FirefoxProfile = 'Andy',
        [Parameter(ValueFromPipeline, Position = 2)]
        [ArgumentCompletions('DIYhome','DIYreddit','DIYexchange','DIYall', 'SelectWithFZF')]
        [Array]$Pages
    )
    Process{[Array]$Pages += @($input)}  
    End{$Pages}
}

With Ask-Web -FirefoxProfile andy -Pages DIYhome,DIYreddit,DIYexchange, in the End block I get :

DIYhome
DIYreddit
DIYexchange

But with 'DIYhome','DIYreddit','DIYexchange'|Ask-Web -FirefoxProfile andy in the End block I get :

DIYexchange
DIYexchange

If I forego all the Begin, Process and End blocks with:

Function Ask-Web{
    [CmdletBinding()]
    Param(
        [Parameter(Position = 1)]
        [ArgumentCompletions('andy','andrew','halley', "hanna")]
        [Array]$FirefoxProfile = 'Andy',
        [Parameter(ValueFromPipeline, Position = 2)]
        [ArgumentCompletions('DIYhome','DIYreddit','DIYexchange','DIYall', 'SelectWithFZF')]
        [Array]$Pages
    )
    $input
}

I get consistent results, all the elements of array $Pages are handed at once, regardless of how the user provided them. How can I do the same but with the Begin, Process and End blocks?

Any help would be greatly appreciated!

Tbh, your use case might not be a great learning example for the pipeline, but if you *really* want to pursue it you should initialise a new *local* variable in your ```BEGIN``` block - e.g. ```$items = @()```, accumulate pipeline items in your ```PROCESS``` block (e.g. ```$items += $_```) and then do something *once* with *all* of the items in your ```END``` block. Note that you should avoid ```+=``` for arrays in production code for performance reasons, but it’s probably fine for your exercise… — mclayton, Apr 10 '23 at 17:05

Santiago Squarzon · Accepted Answer · 2023-04-10T17:49:07.400

You have two options, the recommended one is to collect all input passed through the pipeline with a List<T>, this is done in the process block of your function. Then after all pipeline input has been collected you can start Firefox in your end block.

Main reason to use List<T> to collect pipeline input as opposed to a System.Array (@() and +=) is because arrays are of a fixed size and PowerShell has to recreate it each time we add a new element to it which makes it very inefficient. See this answer and PowerShell scripting performance considerations - Array addition for more details on this.

function Test-Pipeline {
    [CmdletBinding()]
    Param(
        [Parameter(ValueFromPipeline)]
        [Array] $InputObject
    )

    begin {
        $list = [System.Collections.Generic.List[object]]::new()
    }
    process {
        $list.AddRange($InputObject)
    }
    end {
        if($list.Count) {
            # do stuff here with `$list`
            $list.ToArray()
        }
    }
}

Get-ChildItem | Test-Pipeline

The List<T> in this case is necessary because PowerShell will pass one object at a time through the pipeline, this is what's known as One-at-a-time processing. Using $input is not a recommended option in this case because 1. your function is an advanced one and the use of this automatic variable is not recommended in advanced functions. 2. Using $input would mean that your function only works from pipeline only unless you work around it.

The second option, and the not recommended one, is to pass the result of Get-ChildItem through the pipeline without enumerating it. For this you can use Write-Output -NoEnumerate or the comma operator ,:

function Test-Pipeline {
    [CmdletBinding()]
    Param(
        [Parameter(ValueFromPipeline)]
        [Array] $InputObject
    )

    end {
        $InputObject
    }
}

# option 1:
Write-Output (Get-ChildItem) -NoEnumerate | Test-Pipeline
# option 2:
, (Get-ChildItem) | Test-Pipeline

score 1 · Answer 2 · answered Apr 10 '23 at 17:29

Your problem is that you have a scoping problem with your $Pages variable.

When passing via the pipeline, the $Pages variable gets reset to the latest value of the enumeration. This is why you are only getting the last value twice. Basically the Process block sees a new $Pages every time around, so on the last round, you are adding the latest $input to it.

I think the simplest solution is to introduce a new variable that you then use in the End block:

Function Ask-Web{
    [CmdletBinding()]
    Param(
        [Parameter(Position = 1)]
        [ArgumentCompletions('andy','andrew','halley', "hanna")]
        [Array]$FirefoxProfile = 'Andy',
        [Parameter(ValueFromPipeline, Position = 2)]
        [ArgumentCompletions('DIYhome','DIYreddit','DIYexchange','DIYall', 'SelectWithFZF')]
        [Array]$Pages
    )
    Process{
        write-host "Process: $Pages"
        [Array]$AllPages += $Pages
        }  
    End{
        
        $AllPages }
}

How to get all piped in objects at once when using Begin/Process/End blocks?

2 Answers2