Find and Replace Nested JSON Values with Powershell

Question

I have an appsettings.json file that I would like to transform with a PowerShell script in a VSTS release pipeline PowerShell task. (BTW I'm deploying a netstandard 2 Api to IIS). The JSON is structured like the following:

{
    "Foo": {
        "BaseUrl": "http://foo.url.com",
        "UrlKey": "12345"
    },
    "Bar": {
        "BaseUrl": "http://bar.url.com"
    },
    "Blee": {
        "BaseUrl": "http://blee.url.com"
    }
}

I want to replace BaseUrl and, if it exists, the UrlKey values in each section which are Foo, Bar and Blee. (Foo:BaseUrl, Foo:UrlKey, Bar:BaseUrl, etc.)

I'm using the following JSON structure to hold the new values:

{ 
    "##{FooUrl}":"$(FooUrl)", 
    "##{FooUrlKey}":"$(FooUrlKey)",
    "##{BarUrl}":"$(BarUrl)",
    "##{BleeUrl}":"$(BleeUrl)"
}

So far I have the following script:

# Get file path
$filePath = "C:\mywebsite\appsettings.json"

# Parse JSON object from string
$jsonString = "$(MyReplacementVariablesJson)"
$jsonObject = ConvertFrom-Json $jsonString

# Convert JSON replacement variables object to HashTable
$hashTable = @{}
foreach ($property in $jsonObject.PSObject.Properties) {
    $hashTable[$property.Name] = $property.Value
}

# Here's where I need some help

# Perform variable replacements
foreach ($key in $hashTable.Keys) {
    $sourceFile = Get-Content $filePath
    $sourceFile -replace $key, $hashTable[$key] | Set-Content $filePath
    Write-Host 'Replaced key' $key 'with value' $hashTable[$key] 'in' $filePath
}

Answer 1

Why are you defining your replacement values as a JSON string? That's just going to make your life more miserable. If you're defining the values in your script anyway just define them as hashtables right away:

$newUrls = @{
    'Foo'  = 'http://newfoo.example.com'
    'Bar'  = 'http://newbaz.example.com'
    'Blee' = 'http://newblee.example.com'
}

$newKeys = @{
    'Foo' = '67890'
}

Even if you wanted to read them from a file you could make that file a PowerShell script containing those hashtables and dot-source it. Or at least define the values as lists of key=value lines in text files, which can easily be turned into hashtables:

$newUrls = Get-Content 'new_urls.txt' | Out-String | ConvertFrom-StringData
$newKeys = Get-Content 'new_keys.txt' | Out-String | ConvertFrom-StringData

Then iterate over the top-level properties of your input JSON data and replace the nested properties with the new values:

$json = Get-Content $filePath | Out-String | ConvertFrom-Json
foreach ($name in $json.PSObject.Properties) {
    $json.$name.BaseUrl = $newUrls[$name]
    if ($newKeys.ContainsKey($name)) {
        $json.$name.UrlKey = $newKeys[$name]
    }
}
$json | ConvertTo-Json | Set-Content $filePath

Note that if your actual JSON data has more than 2 levels of hierarchy you'll need to tell ConvertTo-Json via the parameter -Depth how many levels it's supposed to convert.

---

Side note: piping the Get-Content output through Out-String is required because ConvertFrom-Json expects JSON input as a single string, and using Out-String makes the code work with all PowerShell versions. If you have PowerShell v3 or newer you can simplify the code a little by replacing Get-Content | Out-String with Get-Content -Raw.

Answer 2

Thank you, Ansgar for your detailed answer, which helped me a great deal. Ultimately, after having no luck iterating over the top-level properties of my input JSON data, I settled on the following code:

$json = (Get-Content -Path $filePath) | ConvertFrom-Json

    $json.Foo.BaseUrl = $newUrls["Foo"]
    $json.Bar.BaseUrl = $newUrls["Bar"]
    $json.Blee.BaseUrl = $newUrls["Blee"]

    $json.Foo.Key = $newKeys["Foo"]

$json | ConvertTo-Json | Set-Content $filePath

I hope this can help someone else.

Answer 3

To update values of keys at varying depth in the json/config file, you can pass in the key name using "." between the levels, e.g. AppSettings.Setting.Third to represent:

{
    AppSettings = {
        Setting = {
            Third = "value I want to update"
        }
    }
}

To set the value for multiple settings, you can do something like this:

$file = "c:\temp\appSettings.json"

# define keys and values in hash table
$settings = @{
    "AppSettings.SettingOne" = "1st value"
    "AppSettings.SettingTwo" = "2nd value"
    "AppSettings.SettingThree" = "3rd value"
    "AppSettings.SettingThree.A" = "A under 3rd"
    "AppSettings.SettingThree.B" = "B under 3rd"
    "AppSettings.SettingThree.B.X" = "Z under B under 3rd"
    "AppSettings.SettingThree.B.Y" = "Y under B under 3rd"
}

# read config file
$data = Get-Content $file -Raw | ConvertFrom-Json

# loop through settings
$settings.GetEnumerator() | ForEach-Object {
    $key = $_.Key
    $value = $_.Value

    $command = "`$data.$key = $value"
    Write-Verbose $command

    # update value of object property
    Invoke-Expression -Command $command
}
        
$data | ConvertTo-Json -Depth 10 | Out-File $file -Encoding "UTF8"

Answer 4

This method deep clones a hashtable but overrides keys which you provide in a separate object. It uses recursion to handle nesting. If you're using PS7 ConvertFrom-Json -AsHashtable is helpful but if you're stuck on PS5 like me you can see another answer of mine for converting the PSObject to a hashtable.

The only downside I've encountered so far is I don't have a decent way of updating an key within an array of objects but I figured I'd share it anyway if I don't get around to or can't find a solution to that.

I dislike the idea of using numeric indexing like foo.0.bar since if the someone inserts another object at the start of the array that statement would update the wrong value so my ideal solution would perform a lookup on another key within the object.

<#
.SYNOPSIS

Given two hashtables this function overrides values of the first using the second.

.NOTES

This function is based on Kevin Marquette's Get-DeepClone
function as documented below.
https://learn.microsoft.com/en-us/powershell/scripting/learn/deep-dives/everything-about-hashtable?view=powershell-7.3#deep-copies

.EXAMPLE
# Use nested hashtable to apply updates.

$Settings = @{
    foo = "foo"
    bar = "bar"
    one = @{
        two = "three"
    }
}

$Updates = @{
    foo = 'fubar'
    one = @{
        two = 'four'
    }
}

$Clone = Update-Hashtable $Settings $Updates
$Clone

.EXAMPLE

# Use flattened hashtable to apply updates.

$Settings = @{
    foo = "foo"
    bar = "bar"
    one = @{
        two = "three"
    }
}

$StringData = @"
foo=fubar
one.two=five
"@
$Updates = $StringData | ConvertFrom-StringData

$Clone = Update-Hashtable $Settings $Updates -UpdatesAreFlattened
$Clone
#>
function Update-Hashtable {
    [CmdletBinding()]
    param(
        [Object] $InputObject,
        [Object] $Updates = @{},
        [Switch] $UpdatesAreFlattened,
        
        # This parameter is used to keep track of our position
        # in a nested object during recursion.
        [Parameter(DontShow)]
        [String] $Keychain
    )
    process {
        if ($InputObject -is [Hashtable]) {
            $Clone = @{}
            
            foreach ($Key in $InputObject.Keys) {
                # Track our nested level by appending keys.
                $Keychain = if ($KeyChain) { "$Keychain.$Key" } else { $Key }

                # Because flattened updates don't keep track our nested level, use the
                # keychain to index it instead of the current key.
                $UpdateKey = if ($UpdatesAreFlattened) { $Keychain } else { $Key }
                $UpdateValue = $Updates.$UpdateKey

                if ($Updates -and $UpdateValue) {
                    $Clone.$Key = $UpdateValue
                } else {
                    # Unflattened updates provide the nested object while flattened updates are single
                    # level so pass the full object
                    $ForwardUpdates = if ($UpdatesAreFlattened) { $Updates } else { $Updates.$Key }
                    $Clone.$Key = Update-Hashtable $InputObject.$Key $ForwardUpdates $Keychain -UpdatesAreFlattened:$UpdatesAreFlattened
                }
                $KeyChain = $null  # Reset the chain.
            }
            return $Clone
        } else {
            return $InputObject
        }
    }
}