2

Powershell seems to tie select cmdlets to the default output stream, and not release it after a command finishes, affecting future command in unexpected ways. It is not consistent in how it does this, which makes generating predictable output impossible without explicitly directing output when writing the script.

This weirdness happens if you put the commands on separate lines, mix in different object generation cmdlets, etc. It doesn't happen if you run the commands on separate lines interactively, and it doesn't happen with objects automatially created on the command line, unless you mix in objects not automatically created on the command line.

I give command line interactive examples, but it happens if you put these commands into a script with each command on a spearate line and then run the script.

PS /home/dennis> get-date|select dayofweek ; get-date

DayOfWeek
---------
   Monday
   Monday

PS /home/dennis> "string1"|select length ; "string2"

Length
------
     7
string2

And for fun, check out this one:

S /home/dennis> "string0" ;"string1"|select length ; get-host ;"string2" ;get-date; 567 ; get-host
string0

Length
------
     7
     1
string2
     1
567
     1

PS /home/dennis> cat test.ps1
"string0"
"string1"|select length
get-host
"string2"
get-date
567
(1..5)
get-host
PS /home/dennis> ./test.ps1
string0

Length
------
     7
     1
string2
     1
567
1
2
3
4
5
     1



...

This also affects objects which are not of the same type, and in fact, it affects objects which do not even have the properties in the select statement. Delaying is not an option, and explictly forcing the output with out-host or write-host will directly write to the powershell output device, making it useless to create a script that will be used to produce objects in a pipeline. It also messes up variables. Observe:

PS /home/dennis> $d = get-date | select dayofweek ; $e = get-date ; $d ; $e

DayOfWeek
---------
   Monday
   Monday

PS /home/dennis> $d

DayOfWeek
---------
   Monday

PS /home/dennis> $e

Monday, August 5, 2019 12:33:47 PM

For those who are thinking, it is only a display issue, and the script can be written to display it correctly, I say again, this makes scripts useless as tools you can reuse in other scripts. Observe how a pipeline inside a script affects commands in an independent interactive shell.

PS /home/dennis> cat test.ps1                    
"string0"
"string1"|select length
get-host
"string2"
get-date
567
get-host

PS /home/dennis> ./test.ps1|% {$_}               
string0

Length
------
     7
     1
string2
     1
567
     1

PS /home/dennis> ./test.ps1|% {write-host $_}
string0
@{Length=7}
System.Management.Automation.Internal.Host.InternalHost
string2
8/5/19 12:50:54 PM
567
System.Management.Automation.Internal.Host.InternalHost
PS /home/dennis> ./test.ps1|% {$_|out-host}  
string0

Length
------
     7


Name             : ConsoleHost
Version          : 6.2.2
InstanceId       : 4e46c643-1a9d-4c55-9151-b311f287a9cb
UI               : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture   : en-US
CurrentUICulture : en-US
PrivateData      : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled  : True
IsRunspacePushed : False
Runspace         : System.Management.Automation.Runspaces.LocalRunspace


string2

Monday, August 5, 2019 1:20:24 PM

567

Name             : ConsoleHost
Version          : 6.2.2
InstanceId       : 4e46c643-1a9d-4c55-9151-b311f287a9cb
UI               : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture   : en-US
CurrentUICulture : en-US
PrivateData      : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled  : True
IsRunspacePushed : False
Runspace         : System.Management.Automation.Runspaces.LocalRunspace

In any shell script I expect that a command will execute independently of the previous command. WhatTheFortran is the logic behind this behaviour? What is the official recommendation to avoid this unpredictability?

Dennis Simpson
  • 127
  • 6
  • 2
    you have run into the `direct vs indirect` host output glitch. PoSh delays all indirect output for ~300ms to see if there is anything "similar" to group. that means all your `Write-Output` and `drop it into the output stream` items will be delayed while the display system checks to see if you have more of the "same type of object" to group with each other. ///// _DIRECT_ output, on the other hand, goes to the screen _right freaking now_. so and `*-Host` stuff will show up immediately - even if that is in the midst of the other "grouped" output. //// continued in next comment ... – Lee_Dailey Aug 05 '19 at 16:25
  • 1
    the solution is to NOT mix the two types, OR to force immediate output with `| Out-Host`, OR to build your indirect output into one object and output that _at once_. OR delay your indirect output long enuf to out-wait the delay. ///// the last two are not recommended ... [*grin*] – Lee_Dailey Aug 05 '19 at 16:28
  • Except it does this with dissimilar objects, and even with objects that have no common properties with each other, or do not have the properties requested in the select at all. – Dennis Simpson Aug 05 '19 at 16:28
  • i suspect the loosey-goosey PoSh type coercion is at work there. i don't know the exact method used, but it looks like some sort of type-fiddling is going on. – Lee_Dailey Aug 05 '19 at 16:31
  • I think it is tied to how powershell decides to create the ouput stream for out-default. I thik it ties select to the stream, not to the commands you put it into. I have updated the question with an example that shows how variables cannot be trusted to display what you think they hold. – Dennis Simpson Aug 05 '19 at 16:35
  • you are now demoing the way that the display formatter changes the **_display_** of values, not what is actually **_in_** the $Vars. [*grin*] – Lee_Dailey Aug 05 '19 at 16:48
  • 1
    That is one way of bringing it, others will say ["it a thing of beauty"](https://stackoverflow.com/a/2042981/1701026). The point is that you assume that the display is default output, but it is not, it is the [pipeline](https://learn.microsoft.com/powershell/scripting/learn/understanding-the-powershell-pipeline?view=powershell-6), only when it is finished (or when specific display commands are used), it outputs to the display. (See: the difference between [`Write-Host` and `Write-Oitput`](https://stackoverflow.com/a/19754384/1701026) – iRon Aug 05 '19 at 16:50
  • If the select is in a script, and you try to use the objects from the script in a separate pipeline, the separate pipeline is affected by the pipeline inside the script. This is not in any way a thing of beauty. It is a thing of confusion and makes it unsuitable as a shell to build tools with. But maybe that isn't what Powershell is for, I suppose. – Dennis Simpson Aug 05 '19 at 17:04
  • I understand that this can be confusing, but it's nothing that hinders building tools. If you build tools properly in PowerShell you output one custom object and additional text as `write-information`, `write-verbose`, `write-warning`, or `write-error`. – vrdse Aug 05 '19 at 17:13
  • This is a common question. Format-table gets set up for one list of properties, but then other sets of properties get hidden. You can see everything if you send it all through format-list. Generally, a script or function will only output one type of object, although get-childitem does something fancy with both file and directory objects. A weird hack is to put get-date at the beginning. Since it has a format file, it seems to make things better. – js2010 Aug 05 '19 at 17:41
  • *WhatTheFortran is the logic behind this behaviour?* When you type anything in PowerShell prompt, PowerShell wrap it like this: `. { Your command here } 2>&1 | Out-Default`. And `Out-Default` responsible for displaying output from your command. Since there is only one `Out-Default` instance per prompt, it is not able to distinguish where one subcommand ends and new one begins. For `Out-Default` it is one continuous input stream. If you do not like how default `Out-Default` work, you can provide your own implementation: `function Out-Default { process { Write-Host "My better thing: $_" } }`. – user4003407 Aug 05 '19 at 18:31
  • Holy cow, I just checked it, and it does always source scripts instead of making a new process for a script. How did I miss that? Thanks PerSetAl, that solves the mystery of why the output stream alteration seemed to carry over to other scripts. They aren't running in their own process and are affected by previous commands in the current process. – Dennis Simpson Aug 07 '19 at 23:15

1 Answers1

0

I know this is confusing. Any time select-object seems to output a table, format-table is actually running in the background. These format commands are kind of an illusion, and controlled by format files. But the object is still the same underneath.

$a = [pscustomobject]@{name='Joe'}
$b = [pscustomobject]@{address='here'}
$a,$b | format-table

name
----
Joe


$a,$b | format-list 

name : Joe

address : here

Other workarounds:

$(get-date|select dayofweek ; get-date) | format-list
$("string1"|select length ; "string2") | format-list
$("string0" ;"string1"|select length ; get-host ;"string2" ;get-date; 567 ; get-host) | format-list
test.ps1 | format-list

The problem with the $d example, is the assignment to $d is only the first statement up to the first semicolon. This is a different issue.

$d = $(get-date | select dayofweek ; $e = get-date ; $d ; $e)
$d | format-list  # 3 objects, unless you repeat the last line

An example like this works, because both commands output object types that have .format.ps1xml files. Once you use select-object, the output is a generic PSCustomObject. Actually, you can try putting get-date first in any script.

get-date;get-host

From Windows Powershell in Action:

Out-Default uses steppable pipelines to run the formatter 
cmdlets [like format-table] to do its rendering and then 
calls Out-Host to display the formatted output.
js2010
  • 23,033
  • 6
  • 64
  • 66
  • The format files seem to travel with the objects. If you generate the objects in one script, and display them in a second script, and then pipe the first script to the second script, the second script will still have the screwed up output. – Dennis Simpson Aug 07 '19 at 10:58
  • No, the format does not travel with the objects. The properties and methods travel with the objects. It's the process that converts these objects into character streams for delivery to a display that gets screwed up. In order for a conversion process to work when more than one type of object is in the input stream, the converter would have to be polymorphic. The existing converter isn't. – Walter Mitty Aug 07 '19 at 13:25
  • @WalterMitty There are cases where format-table can handle different object types in one shot, like with get-childitem directories and files, but the format files are setup for that. – js2010 Aug 07 '19 at 13:28
  • Thanks, @js2010. Those format files are built by the PS Architects, are they not? Is it possible for a user to come up with a format file that accommodates more than one format? If so, and if it isn't too much of a hassle, that may be the answer that Dennis is really looking for. – Walter Mitty Aug 07 '19 at 13:40
  • @WalterMitty It's possible, but I wouldn't call it for beginners. They are xml files. – js2010 Aug 07 '19 at 14:39
  • Dennis doesn't sound like a beginner to me. He may be new to powershell, but it seems as though he's been around the technology for a while. – Walter Mitty Aug 07 '19 at 15:17
  • Yeah, not too interested in playing with format files, it again misses the point I was trying to get at, that powershell has this built-in assumption that a script only should generate objects of one type. I understand why that assumption breaks output if you don't code for the assumption, but I don't understand why powershell was designed that way. I have only cynical guesses based on quite a few years of having to adapt what I want to do to the Microsoft idea of what I should want to do. – Dennis Simpson Aug 08 '19 at 12:29
  • The problem with really with format-table vs format-list. – js2010 Aug 08 '19 at 12:31
  • It would be helpful if you would tell us what you are trying to do with Powershell. The examples you have given illustrate PS behavior that you do not want. But that doesn't help us to help you with PS. Perhaps you could show us a script you are having trouble with, one where you implemented something similar in some language like Python. Then maybe we could help you deal with the way PS works. – Walter Mitty Aug 09 '19 at 15:55
  • This investigation started when I had a script that did not work as expected (see below). I already knew every command could be explictly piped to out-host, but the behaviour was unexpected and I wanted to know what implications that had for producing output other than simple text reports with Powershell. I believe I have a good grasp of the issues now (unpredictable output requiring explicit output control for all commands and all scripts being sourced). gwmi win32_operatingsystem|select registereduser,version,manufacturer; gwmi win32_bios; gwmi win32_computersystem – Dennis Simpson Aug 10 '19 at 00:59
  • I will say powershell appears to be more predictable with output if you ensure you only produce one object type as output from a script. – Dennis Simpson Aug 10 '19 at 01:04