3

Long story short - it seems impossible to get a pipeline value to resolve to plain text in a ScriptProperty and so I'm looking for a way to either do that, or to somehow parameterize when I'm adding properties to an object. I would be perfectly happy to add the values as NotePropertyMembers rather than a ScriptProperty; but I can't find a way to do that without looping through the entire object multiple times. The advantage of the ScriptProperty is the ability to use the $this variable so that I don't need to call Add-Member on each member of the object itself.

Here's what an example of what I'm trying to accomplish and where it is breaking down.

# This sample file contains the values that I'm interested in
@'
ID,CATEGORY,AVERAGE,MEDIAN,MIN,MAX
100,"",52,50,10,100
100,1,40,40,20,60
100,2,41,35,15,85
'@ > values.csv

# To access the values, I decided to create a HashTable
$map = @{}
(Import-Csv values.csv).ForEach({$map[$_.ID + $_.CATEGORY] = $_})

# This is my original object. In reality, it has several additional columns of data - but it is
# missing the above values which is why I want to pull them in
@'
ID,CATEGORY
100,""
100,1
100,2
'@ > object.csv

$object = Import-Csv object.csv

# This is the meat of my attempt. Loop through all of the properties in the HashTable that
# I need to pull into my object and add them
$map.Values | Get-Member -MemberType NoteProperty | Where-Object {$_.Name -NotIn 'ID', 'CATEGORY'} | ForEach-Object {$object | Add-Member -MemberType ScriptProperty -Name "$($_.Name)" -Value {$map[$this.ID + $this.CATEGORY]."$($_.Name)"}}

In theory, I know that it works because I can display the values for a particular member of my object using the code below

$map.Values | Get-Member -MemberType NoteProperty | Where-Object {$_.Name -NotIn 'ID', 'CATEGORY'} | ForEach-Object {"$($_.Name) = " + $map[$object[0].ID + $object[0].CATEGORY]."$($_.Name)"}

Displays

AVERAGE = 52
MAX = 100
MEDIAN = 50
MIN = 10

However - it is clear that $_.Name and "$($_.Name)" do not resolve to their plain text description when they're in the ScriptProperty

PS> $object | GM -MemberType ScriptProperty

# Displays
   TypeName: System.Management.Automation.PSCustomObject

Name    MemberType     Definition
----    ----------     ----------
AVERAGE ScriptProperty System.Object AVERAGE {get=$map[$this.ID + $this.CATEGORY]."$($_.Name)";}
MAX     ScriptProperty System.Object MAX {get=$map[$this.ID + $this.CATEGORY]."$($_.Name)";}
MEDIAN  ScriptProperty System.Object MEDIAN {get=$map[$this.ID + $this.CATEGORY]."$($_.Name)";}
MIN     ScriptProperty System.Object MIN {get=$map[$this.ID + $this.CATEGORY]."$($_.Name)";}

To me, an obvious workaround is to individually add each column which still allows me to take advantage of using Add-Member on the object itself instead of each individual member. However, the entire idea is to be able to dynamically add values without knowing ahead of time what their names are - in order to do that, I need to find a way to force the name to resolve within the ScriptProperty

# Workaround for this incredibly simple example
PS> $object | Add-Member -MemberType ScriptProperty -Name AVERAGE -Value {$map[$this.ID + $this.CATEGORY]."AVERAGE"}
PS> $object | Add-Member -MemberType ScriptProperty -Name MEDIAN -Value {$map[$this.ID + $this.CATEGORY]."MEDIAN"}
PS> $object | Add-Member -MemberType ScriptProperty -Name MIN -Value {$map[$this.ID + $this.CATEGORY]."MIN"}
PS> $object | Add-Member -MemberType ScriptProperty -Name MAX -Value {$map[$this.ID + $this.CATEGORY]."MAX"}

PS> $object | GM -MemberType ScriptProperty

# Displays

   TypeName: System.Management.Automation.PSCustomObject

Name    MemberType     Definition
----    ----------     ----------
AVERAGE ScriptProperty System.Object AVERAGE {get=$map[$this.ID + $this.CATEGORY]."AVERAGE";}
MAX     ScriptProperty System.Object MAX {get=$map[$this.ID + $this.CATEGORY]."MAX";}
MEDIAN  ScriptProperty System.Object MEDIAN {get=$map[$this.ID + $this.CATEGORY]."MEDIAN";}
MIN     ScriptProperty System.Object MIN {get=$map[$this.ID + $this.CATEGORY]."MIN";}
immobile2
  • 489
  • 2
  • 15
  • The end result is a merge of `values.csv` and `object.csv` having the same `ID` + `CATEGORY` property values ? – Santiago Squarzon Aug 26 '22 at 14:45
  • @SantiagoSquarzon - precisely – immobile2 Aug 26 '22 at 15:02
  • Not if understand the hassle with the `ScriptProperty`,, so far I can tell, your question effectively comes down to this question: [In PowerShell, what's the best way to join two tables into one?](https://stackoverflow.com/q/1848821/1701026) (or any of the related ones in my answer). Using the [Join-Script](https://www.powershellgallery.com/packages/Join-Object) from my answer: `Import-Csv object.csv |Update-Object (Import-Csv values.csv) -on ID,CATEGORY`. – iRon Aug 26 '22 at 16:59
  • 1
    Be aware that there is even a pitfall in your approach where two different ID,CATEGORY might resolve in the same key (e.g. `12,1` and `1,21`). See: [Does there exist a designated (sub)index delimiter?](https://stackoverflow.com/q/72772327/1701026) – iRon Aug 26 '22 at 16:59

4 Answers4

2

Use Select-Object instead of Add-Member:

$object |Select *,@{Name='Average';Expression={$map[$_.ID + $_.Category].Average}},@{Name='Median';Expression={$map[$_.ID + $_.Category].Median}},@{Name='Min';Expression={$map[$_.ID + $_.Category].Min}},@{Name='Max';Expression={$map[$_.ID + $_.Category].Max}}

Select-Object will evaluate the Expression block against every single individual input item, thereby resolving the mapping immediately, as opposed to deferring it until someone references the property

Mathias R. Jessen
  • 157,619
  • 12
  • 148
  • 206
  • Issue with this method, aside from the fact that I initially planned to defer evaluation until using `Export-Csv`, is that it seems like you'd need to know the names of every new property you're selecting. The original goal was to loop through the `NoteProperty` members of $values and all of them without necessarily knowing their names or even the total count – immobile2 Aug 26 '22 at 15:14
  • @immobile2 I must be missing something here? - if you want to recreate the objects in `$values` then all you need is `$valuesCopied = $values |Select *` – Mathias R. Jessen Aug 26 '22 at 15:18
  • 1
    My `$object` has more columns than just the ID and Category, for the sake of simplicity I just didn't add all of those. So the goal is to merge the columns from `$values` into `$object` keeping all of the original columns contained in `$object` – immobile2 Aug 26 '22 at 15:21
2

Complementing the helpful answer from Mathias which creates new objects, here is how you can approach dynamically updating the object itself:

$exclude = 'ID', 'CATEGORY'
$map = @{}
$properties = $values[0].PSObject.Properties.Name | Where-Object { $_ -notin $exclude }
$values.ForEach{ $map[$_.ID + $_.CATEGORY] = $_ | Select-Object $properties }

foreach($line in $object) {
    $newValues = $map[$line.ID + $line.CATEGORY]
    foreach($property in $properties) {
        $line.PSObject.Properties.Add(
            [psnoteproperty]::new($property, $newValues.$property)
        )
    }
}

Above code assumes that both Csvs ($values and $object) are loaded in memory and that you know which properties from the $values Csv should be excluded.

mklement0
  • 382,024
  • 64
  • 607
  • 775
Santiago Squarzon
  • 41,465
  • 5
  • 14
  • 37
  • 1
    Perfect! I think this is the route I was thinking in terms of looping through `$object`, but the nested `ForEach` is what I was missing so that I didn't need to loop through the entire thing multiple times! – immobile2 Aug 26 '22 at 15:29
2

One additional option that I stumbled upon while working with Santiago's great solution. Using Add-Member rather than the Properties.Add() method would allow for a hash table of NotePropertyMembers to used, which could be advantageous although I haven't started testing performance of different methods yet. It makes things a bit simpler if you already have a hash table of the properties and values that you want to add to your object.

$map = @{}

# This will create a hash table nested in the hash table, rather than a hash table whose values are PSCustomObjects
$values.ForEach{$map[$_.ID + $_.CATEGORY] = $_.PSObject.Properties | ?{$_.Name -notin $exclude} | % {$ht = @{}} {$ht[$_.Name] = $_.Value} {$ht}}

ForEach($line in $object){
    If($map.Contains($line.ID + $line.CATEGORY){
        $line | Add-Member -NotePropertyMembers $map[$line.ID + $line.CATEGORY]
    }
}
immobile2
  • 489
  • 2
  • 15
  • Good point to reduce the code but note, what `Add-Member` is doing behind the scenes is very similar to what my code does :) – Santiago Squarzon Aug 26 '22 at 17:44
  • 2
    Agreed - and it is definitely nice to see them both. The ability to add a hash table in one swoop is potentially helpful for others, but based on some testing I've done with the files I was working with, it looks like your method outperforms the overhead of calling `Add-Member` and also throws fewer errors in instances where the lookup value doesn't exist as a key in the hash table (throwing errors may be better, TBD!) – immobile2 Aug 26 '22 at 20:13
2

Santiago's answer and your own, based on static NoteProperty members rather than the originally requested dynamic ScriptProperty members, turned out to be the better solution in the case at hand.


As for how you could have made your dynamic ScriptProperty approach work:

$map.Values | 
  Get-Member -MemberType NoteProperty | 
  Where-Object Name -NotIn ID, CATEGORY | 
  ForEach-Object {
    # Cache the property name at hand, so it can be referred to
    # in the pipeline below.
    $propName = $_.Name
    # .GetNewClosure() is necessary for capturing the value of $amp
    # as well as the current value of $propName as part of the script block.
    $object |
      Add-Member -MemberType ScriptProperty `
                 -Name $propName `
                 -Value { $map[$this.ID + $this.CATEGORY].$propName }.GetNewClosure()
  }

Note that use of .GetNewClosure() is crucial, so as to capture:

  • each iteration's then-current $propName variable value
  • the $map variable's hashtable

as part of the script block.

Note:

  • .GetNewClosure() is expensive, as it creates a dynamic module behind the scenes in which the captured values are cached and to which the script block gets bound.

  • Generally, it's better to restrict ScriptProperty members to calculations that are self-contained, i.e. rely only on the state of the object itself (accessible via automatic $this variable), not also on outside values; a simple example:

    $o = [pscustomobject] @{ A = 1; B = 2}
    $o | Add-Member -Type ScriptProperty -Name C { $this.A + $this.B }
    $o.C # -> 3
    $o.A = 2; $o.C # -> 4
    
mklement0
  • 382,024
  • 64
  • 607
  • 775