40

How do you send the output from one CmdLet to the next one in a pipeline as a complete array-object instead of the individual items in the array one at a time?

The problem - Generic description
As can be seen in help for about_pipelines (help pipeline) powershell sends objects one at the time down the pipeline¹. So Get-Process -Name notepad | Stop-Process sends one process at the time down the pipe.

Lets say we have a 3rd party CmdLet (Do-SomeStuff) that can't be modified or changed in any way. Do-SomeStuff perform different if it is passed an array of strings or if it is passed a single string object.

Do-SomeStuff is just an example, it could be substituted for ForEach-Object, Select-Object, Write-Host (or any other CmdLet accepting pipeline input)

Do-SomeStuff will in this example process the individual items in the array one at the time.

$theArray = @("A", "B", "C")
$theArray | Do-SomeStuff

If we want to send the complete array as one object to Do-SomeStuff one might try something like this

@($theArray) | Do-SomeStuff

But it does not produce the expected result since PowerShell "ignores" the new single-item-array.

So, how do you "force" $theArray to be passed down the pipe as a single array-object instead of the content items one at the time?


The problem - practical example
As shown below the output of Write-Host is different if passed an array or if it passed the individual items in the array one at the time.

PS C:\> $theArray = @("A", "B", "C")
PS C:\> Write-Host $theArray
A B C
PS C:\> $theArray | foreach{Write-Host $_}
A
B
C
PS C:\> @($theArray) | foreach{Write-Host $_}
A
B
C

How do you do to get $theArray | foreach{Write-Host $_} to produce the same output as Write-Host $theArray ?




FOOTNOTES

  1. Pipeline processing in Powershell

A normal array of strings

PS C:\> @("A", "B", "C").GetType().FullName
System.Object[]


A normal array of strings piped to Foreach-Object

PS C:\> @("A", "B", "C") | foreach{$_.GetType().FullName}
System.String
System.String
System.String

Each string in the array is processed one at the time by the ForEach-Object CmdLet.


An array of arrays, where the "inner" arrays are arrays of strings.

PS C:\> @(@("A", "B", "C"), @("D", "E", "F"), @("G", "H", "I")) | foreach{$_.GetType().FullName}
System.Object[]
System.Object[]
System.Object[]

Each array in the array is processed one at the time by the ForEach-Object CmdLet, and the content of each sub-array from the input is handled as one object even though it is an array.

NoOneSpecial
  • 695
  • 1
  • 6
  • 16
  • 3
    Powershell unrolls arrays. There are discussions about that on this site and other places. The unrolling is weird but means you can't `@(@(@()))` and get more than a single array (roughly). – Etan Reisner Apr 30 '15 at 16:40
  • 10
    Ask **one** question please. Remove all the other extraneous stuff (none of which has anything to do with your original question) and your original question is the one you least explained. Does wrapping the output in `@()` not work for that? – Etan Reisner Apr 30 '15 at 16:41
  • 2
    I agree with @EtanReisner, this is very hard to follow. If you have multiple questions, post them separately. Even if some of the supporting examples overlap, it's better to do it that way. Also as mentioned, powershell unrolls arrays. It also tries to coerce the right side of a comparison which is why swapping the objects can give you different results. – briantist Apr 30 '15 at 17:03
  • what are you trying to do? `"A", "B", "C" | Format-Custom -Expand CoreOnly` proves you are pushing whole array to the `Format-Custom` cmdlt, so what do you want? – Vincent De Smet Apr 30 '15 at 22:30
  • Clarified, elaborated and concretized the question – NoOneSpecial May 01 '15 at 10:43

6 Answers6

48

Short answer: use unary array operator ,:

,$theArray | foreach{Write-Host $_}

Long answer: there is one thing you should understand about @() operator: it always interpret its content as statement, even if content is just an expression. Consider this code:

$a='A','B','C'
$b=@($a;)
$c=@($b;)

I add explicit end of statement mark ; here, although PowerShell allows to omit it. $a is array of three elements. What result of $a; statement? $a is a collection, so collection should be enumerated and each individual item should be passed by pipeline. So result of $a; statement is three elements written to pipeline. @($a;) see that three elements, but not the original array, and create array from them, so $b is array of three elements. Same way $c is array of same three elements. So when you write @($collection) you create array, that copy elements of $collection, instead of array of single element.

user4003407
  • 21,204
  • 4
  • 50
  • 60
  • 3
    Thanks, now when you told me what to look for it is all in the documentation [TechNet: about_Operators](https://technet.microsoft.com/en-us/library/hh847732.aspx) "_, Comma operator As a binary operator, the comma creates an array. As a unary operator, the comma creates an array with one member. Place the comma before the member._ `$myArray = 1,2,3 $SingleArray = ,1`" – NoOneSpecial May 01 '15 at 10:50
  • 1
    @NoOneSpecial I update my answer with details of `@()` operator behavior. – user4003407 May 01 '15 at 11:02
  • So what is the equivalent of C#: `object[] myArray = new object[] { new object[0] };`? – NoOneSpecial May 01 '15 at 11:20
  • 2
    @NoOneSpecial `$array=,@()` – user4003407 May 01 '15 at 11:22
  • @PetSerAl thanks for helpful answer. I believe there is nothing special about `@()` operator in your Long Answer part. The `()` expression demonstrates completely the same behavior in provided examples (at least in Powershell v5+) – maoizm Mar 26 '18 at 03:33
  • @maoizm First: `($a;)` is not syntactically valid. Second: `$a='A','B','C'; $b=($a); $c=@($a); [Object]::ReferenceEquals($a, $b) <#True#>; [Object]::ReferenceEquals($a, $c) <#False#>` – user4003407 Mar 26 '18 at 03:39
  • @PeterSerAl I do not say `@()` and `()` are identical, I say is that `()` unwraps array in the same fashion: `$a|%{$_.GetType().FullName}`, `$b|%{$_.GetType().FullName}` and `$c|%{$_.GetType().FullName}` produce the same result: `System.String System.String System.String` – maoizm Mar 26 '18 at 04:49
  • @maoizm `()` does not unwrap arrays. – user4003407 Mar 26 '18 at 07:03
  • @PetSerAl DUDE!!! That comment about the `@()` operator always interpreting its content as statement was so eye-opening!!! I tend to bumble along in PS somewhat, but that made so many code snippets suddenly make sense. – Chiramisu Aug 30 '18 at 23:32
10

The comma character makes the data an array. In order to make the pipe line process your array as an array, instead of operating on each array element individually, you may also need to wrap the data with parentheses.

This is useful if you need to assess the status of multiple items in the array.

Using the following function

function funTest {
    param (
        [parameter(Position=1, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
        [alias ("Target")]
        [array]$Targets 
        ) # end param
    begin {}
    process {
        $RandomSeed = $( 1..1000 | Get-Random )
        foreach ($Target in $Targets) {
            Write-Host "$RandomSeed - $Target"
            } # next target
        } # end process
    end {}
    } # end function 

Consider the following examples:

Just Wrapping your array in parentheses does not guarantee the function will process the array of values in one process call. In this example we see the random number changes for each element in the array.

PS C:\> @(1,2,3,4,5) | funTest
153 - 1
87 - 2
96 - 3
96 - 4
986 - 5

Simply adding the leading comma, also does not guarantee the function will process the array of values in one process call. In this example we see the random number changes for each element in the array.

PS C:\> , 1,2,3,4,5 | funTest
1000 - 1
84 - 2
813 - 3
156 - 4
928 - 5

With the leading comma and the array of values in parentheses, we can see the random number stays the same because the function's foreach command is being leveraged.

PS C:\> , @( 1,2,3,4,5) | funTest
883 - 1
883 - 2
883 - 3
883 - 4
883 - 5
Ro Yo Mi
  • 14,790
  • 5
  • 35
  • 43
5

There's an old-school solution, if you don't mind that your process is a function.

Example: You want an array copied to the clipboard in a way that allows you to build it again on another system without any PSRemoting connectivity. So you want an array containing "A", "B", and "C" to transmute to a string: @("A","B","C") ...instead of a literal array.

So you build this (which isn't optimal for other reasons, but stay on topic):

# Serialize-List

param 
(
    [Parameter(Mandatory, ValueFromPipeline)]
    [string[]]$list
)
    $output = "@(";

    foreach ($element in $list)
    {
        $output += "`"$element`","
    }

    $output = $output.Substring(0, $output.Length - 1)
    $output += ")"
    $output

and it works when you specify the array as a parameter directly:

Serialize-List $list
@("A","B","C")

...but not so much when you pass it through the pipeline:

$list | Serialize-List
@("C")

But refactor your function with begin, process, and end blocks:

# Serialize-List

param 
(
    [Parameter(Mandatory, ValueFromPipeline)]
    [string[]]$list
)

begin
{
    $output = "@(";
}

process
{
    foreach ($element in $list)
    {
        $output += "`"$element`","
    }
}

end
{
    $output = $output.Substring(0, $output.Length - 1)
    $output += ")"
    $output
}

...and you get the desired output both ways.

Serialize-List $list
@("A","B","C")

$list | Serialize-List
@("A","B","C")
MarredCheese
  • 17,541
  • 8
  • 92
  • 91
Niali
  • 94
  • 1
  • 5
2

The most "correct" way would be to use the Write-Output cmdlet and specify the -NoEnumerate switch:

Write-Output $theArray -NoEnumerate | Do-SomeStuff

Also, the author states:

I have a second way that's more of a hack (and I try to avoid hacks like this). You can place a comma in front of the array before you pipe it.

Both will work, but using the comma operator will always create an additional array to contain the original one, whereas Write-Output -NoEnumerate will write the original array to the pipeline in one step.

marsze
  • 15,079
  • 5
  • 45
  • 61
  • 1
    Please elaborate on why `Write-Output` is more correct than using the unary array operator. – NoOneSpecial Oct 26 '21 at 13:38
  • 1
    @NoOneSpecial see my edit – marsze Oct 26 '21 at 20:04
  • "In one step" ? `Write-Output` has to receive the array then uses `WriteObject(arr, false)` to not enumerate the output: `& {[CmdletBinding()]param($a) $PSCmdlet.WriteObject($a, $false) } (0..10) | % { "[$_]" }` vs `& {param($a) ($arr = [array]::CreateInstance([Object], 1))[0] = $a; $arr } (0..10) | % { "[$_]" }` there isnt any difference. Just look at the source code... https://github.com/PowerShell/PowerShell/blob/0d6b93a23f92d3f39cd92eeaa7934d6d6778f089/src/System.Management.Automation/engine/DefaultCommandRuntime.cs#L60 – Santiago Squarzon Mar 04 '23 at 19:30
  • @SantiagoSquarzon There is a difference, see `Write-Output @(1, 2, 3) | % { $_.GetType() }` vs `Write-Output @(1, 2, 3) -NoEnumerate | % { $_.GetType() }` – marsze Mar 05 '23 at 20:55
  • I wasn't comparing `Write-Output` with `Write-Output -NoEnumerate`, I was comparing `Write-Output -NoEnumerate` with the __comma operator__, which according to your answer the latter creates a new array before sending the output down the pipeline and the former does not when its not really the case. Both do something very similar excpet one does so behind the scenes. – Santiago Squarzon Mar 05 '23 at 20:58
  • @SantiagoSquarzon I am saying, using the comma operator is a hack. A hack that works, because it basically sends an array containing an array to the pipeline, so enumerating results in the original array. – marsze Mar 05 '23 at 21:01
  • Yes indeed, but `Write-Output -NoEnumerate` is also a "hack" in that case, just that you dont know whats happening behind the scenes. – Santiago Squarzon Mar 05 '23 at 21:02
  • @SantiagoSquarzon The whole point of this answers is how, according to PowerShell docs, would be the "right" way to accomplish this, but feel free to use whatever works for you ;) – marsze Mar 05 '23 at 21:04
  • @SantiagoSquarzon Btw, I don't see how `Write-Output -NoEnumerate` is a "hack". Of course it receives the array, what else should happen? But then, it does not enumerate it. Vs the comma operator would wrap it into a *new* array before sending it to the pipeline. – marsze Mar 05 '23 at 21:09
  • The source code is showing us that what the comma operator is doing and the cmdlet are doing are very similar except the cmdlet adds the collection to a List and sends it down the pipeline. So behind the scenes they're doing almost the same. I'm not arguing about "what is more correct to use from a pwsh perspective", I'm saying that what you're stating in your answer is not accurate. – Santiago Squarzon Mar 05 '23 at 21:14
  • @SantiagoSquarzon Are you talking about the input object being added to an output "list"? Do not confuse what the source code is doing with the overhead using the comma operator involes, which is creating a _new_ array which then is enumerated to be sent to the pipeline, no matter how that is implemented. – marsze Mar 05 '23 at 21:49
  • Yeah, I'm done arguing. You should know that each time you call `Write-Output` that List is being created then the binder is called to assign that input to the list and then the cmdlet calls `WriteObject` with the `noenumerate` argument. If that makes you think there is __less overhead__ than creating a single element array and adding reference type to it not sure what to tell you. – Santiago Squarzon Mar 05 '23 at 22:00
  • @SantiagoSquarzon You should realize that what you are explaining happens *every* time as part of the normal pipeline implementation. And that the commare operation creates *additional* overhead, on top of that. I'm not sure what your point is. – marsze Mar 05 '23 at 22:02
2

Implementation from function usage

Write-Output -NoEnumerate

Write-Output 1, 2.2, '3' -NoEnumerate | Get-Member -Name GetType

Implementation from function definition

Put the code of the Process block in the End block.

function PipelineDemoA {
    param (
        [Parameter(ValueFromPipeline)]
        [String[]]$Value = '.'
    )
    begin {
        Write-Output '----------begin'
        # $valueList = @() # Object[] cannot add objects dynamically
        $valueList = [System.Collections.ArrayList]@()
    }
    process {
        Write-Output 'process'
        $valueList.Add($Value) | Out-Null
    }
    end {
        Write-Output 'end'
        $Value = $PSBoundParameters['Value'] = $valueList
        Write-Output ($Value -join ', ')
    }
}

'A', 'B' |  PipelineDemoA
@() | PipelineDemoA
PipelineDemoA

Use Automatic Variable $input.

function PipelineDemoB {
    param (
        [Parameter(ValueFromPipeline)]
        [String[]]$Value
    )
    if ($input.Count -gt 0) { $Value = $PSBoundParameters['Value'] = $input }
}

'A', 'B', 'C' | PipelineDemoB

There is a problem with this method that it cannot distinguish between the two calling methods. Not recommended for parameters with default values.

@() | PipelineDemoB
PipelineDemoB

# What will happen?
@() | Get-ChildItem # -Path is @()
Get-ChildItem # -Path is default value '.'

About $input

In a function without param block, the $input variable is ArrayListEnumeratorSimple.

In a function with param block, and in the begin block, the $input variable is ArrayList[0].

In a function with param block, and in the process block, the $input variable is ArrayList[1].

In a function with param block, and in the end block, the $input variable is Object[0].

In a function with param block, and without a begin, process, end block, the $input variable is Object[].

function PipelineDemo1 {
    begin {
        Write-Output '----------begin'
        Write-Output "$($input.GetType().Name) / $($input.MoveNext()) / $($input.Current)"
    }
    process {
        Write-Output '----------process'
        Write-Output "$($input.GetType().Name) / $($input.MoveNext()) / $($input.Current)"
    }
    end {
        Write-Output '----------end'
        Write-Output "$($input.GetType().Name) / $($input.MoveNext()) / $($input.Current)"
    }
}

'A', 'B', 'C' | PipelineDemo1 -Z 'Z'

function PipelineDemo2 {
    param (
        [Parameter(ValueFromPipeline)]
        [String[]]$Value,
        [string]$Z
    )
    begin {
        Write-Output '----------begin'
        Write-Output "$($input.GetType().Name) / $($input.Count) / $($input -join ', ')"
    }
    process {
        Write-Output '----------process'
        Write-Output "$($input.GetType().Name) / $($input.Count) / $($input -join ', ')"
    }
    end {
        Write-Output '----------end'
        Write-Output "$($input.GetType().Name) / $($input.Count) / $($input -join ', ')" # $input = Object[0]
    }
}

'A', 'B', 'C' | PipelineDemo2 -Z 'Z'

function PipelineDemo3 {
    param (
        [Parameter(ValueFromPipeline)]
        [String[]]$Value,
        [string]$Z
    )
    Write-Output '----------default'
    Write-Output "$($input.GetType().Name) / $($input.Count) / $($input -join ', ')"
}

'A', 'B', 'C' | PipelineDemo3 -Z 'Z'

function PipelineDemoValue {
    param (
        [string]$Tag,
        [Parameter(ValueFromPipeline)]
        [String[]]$Value = '.'
    )
    
    Write-Output "----------$Tag"
    Write-Output "Value = $($Value -join ', ') / PSValue = $($PSBoundParameters.ContainsKey('Value')) / $($PSBoundParameters['Value'] -join ', ')"
    Write-Output "input = $($input.Count) / $($input -join ', ')"
}

'A', 'B', 'C' | PipelineDemoValue -Tag 1
@() | PipelineDemoValue -Tag 2
$null | PipelineDemoValue -Tag 3
PipelineDemoValue -Value 'A', 'B', 'C' -Tag 4
PipelineDemoValue -Value $null -Tag 5
PipelineDemoValue -Tag 6
ZSkycat
  • 9,372
  • 2
  • 8
  • 5
0
$ar="1","2","3"

$ar | foreach { $_ }
Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
  • 3
    While this code snippet may solve the problem, it doesn't explain why or how it answers the question. Please [include an explanation for your code](https://meta.stackexchange.com/q/114762/269535), as that really helps to improve the quality of your post. Remember that you are answering the question for readers in the future, and those people might not know the reasons for your code suggestion. You can use the [edit] button to improve this answer to get more votes and reputation! – Brian Tompsett - 汤莱恩 Aug 25 '20 at 08:04
  • 1
    In what way does this solve the problem? `$ar | foreach { $_ }` is pretty much the same as just `$ar`. It does not process the array as one object like `Write-Host $ar` do. – NoOneSpecial Aug 26 '20 at 08:19