Convert JSON to a PowerShell Object, then, replace that Object with a new Object and convert back to JSON

Question

Background

I have some JSON for a Packer build like so:

{
  "builders": [
    {
      "type": "azure-arm"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/docker-compose.sh",
        "{{template_dir}}/scripts/installers/docker-moby.sh"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}",
        "DOCKERHUB_LOGIN\u003d{{user `dockerhub_login`}}",
        "DOCKERHUB_PASSWORD\u003d{{user `dockerhub_password`}}"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} {{ .Path }}\u0027"
    },
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/azcopy.sh",
        "{{template_dir}}/scripts/installers/azure-cli.sh",
        "{{template_dir}}/scripts/installers/azure-devops-cli.sh",
        "{{template_dir}}/scripts/installers/basic.sh",
        "{{template_dir}}/scripts/installers/bicep.sh",
        "{{template_dir}}/scripts/installers/aliyun-cli.sh",
        "{{template_dir}}/scripts/installers/apache.sh",
        "{{template_dir}}/scripts/installers/aws.sh",
        "{{template_dir}}/scripts/installers/clang.sh",
        "{{template_dir}}/scripts/installers/swift.sh",
        "{{template_dir}}/scripts/installers/cmake.sh",
        "{{template_dir}}/scripts/installers/codeql-bundle.sh",
        "{{template_dir}}/scripts/installers/containers.sh",
        "{{template_dir}}/scripts/installers/dotnetcore-sdk.sh",
        "{{template_dir}}/scripts/installers/erlang.sh",
        "{{template_dir}}/scripts/installers/firefox.sh",
        "{{template_dir}}/scripts/installers/gcc.sh",
        "{{template_dir}}/scripts/installers/gfortran.sh",
        "{{template_dir}}/scripts/installers/git.sh",
        "{{template_dir}}/scripts/installers/github-cli.sh",
        "{{template_dir}}/scripts/installers/google-chrome.sh",
        "{{template_dir}}/scripts/installers/google-cloud-sdk.sh",
        "{{template_dir}}/scripts/installers/haskell.sh",
        "{{template_dir}}/scripts/installers/heroku.sh",
        "{{template_dir}}/scripts/installers/hhvm.sh",
        "{{template_dir}}/scripts/installers/java-tools.sh",
        "{{template_dir}}/scripts/installers/kubernetes-tools.sh",
        "{{template_dir}}/scripts/installers/oc.sh",
        "{{template_dir}}/scripts/installers/leiningen.sh",
        "{{template_dir}}/scripts/installers/miniconda.sh",
        "{{template_dir}}/scripts/installers/mono.sh",
        "{{template_dir}}/scripts/installers/kotlin.sh",
        "{{template_dir}}/scripts/installers/mysql.sh",
        "{{template_dir}}/scripts/installers/mssql-cmd-tools.sh",
        "{{template_dir}}/scripts/installers/sqlpackage.sh",
        "{{template_dir}}/scripts/installers/nginx.sh",
        "{{template_dir}}/scripts/installers/nvm.sh",
        "{{template_dir}}/scripts/installers/nodejs.sh",
        "{{template_dir}}/scripts/installers/bazel.sh",
        "{{template_dir}}/scripts/installers/oras-cli.sh",
        "{{template_dir}}/scripts/installers/phantomjs.sh",
        "{{template_dir}}/scripts/installers/php.sh",
        "{{template_dir}}/scripts/installers/postgresql.sh",
        "{{template_dir}}/scripts/installers/pulumi.sh",
        "{{template_dir}}/scripts/installers/ruby.sh",
        "{{template_dir}}/scripts/installers/r.sh",
        "{{template_dir}}/scripts/installers/rust.sh",
        "{{template_dir}}/scripts/installers/julia.sh",
        "{{template_dir}}/scripts/installers/sbt.sh",
        "{{template_dir}}/scripts/installers/selenium.sh",
        "{{template_dir}}/scripts/installers/smallstep-cli.sh",
        "{{template_dir}}/scripts/installers/terraform.sh",
        "{{template_dir}}/scripts/installers/packer.sh",
        "{{template_dir}}/scripts/installers/vcpkg.sh",
        "{{template_dir}}/scripts/installers/dpkg-config.sh",
        "{{template_dir}}/scripts/installers/mongodb.sh",
        "{{template_dir}}/scripts/installers/android.sh",
        "{{template_dir}}/scripts/installers/yq.sh",
        "{{template_dir}}/scripts/installers/pypy.sh",
        "{{template_dir}}/scripts/installers/python.sh",
        "{{template_dir}}/scripts/installers/graalvm.sh"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}",
        "DEBIAN_FRONTEND\u003dnoninteractive"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} {{ .Path }}\u0027"
    },
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/Install-Toolset.ps1",
        "{{template_dir}}/scripts/installers/Configure-Toolset.ps1"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} pwsh -f {{ .Path }}\u0027"
    }
  ]
}

I am trying to add some items inside the provisioners list, but only inside the second index list called scripts, so in this case the scripts object. Ideally doing this all in PowerShell (I am using PowerShell 7 latest).

What I have tried

Doing some Googling around, it was suggested that I convert the JSON to a PowerShell Object, and then attempt to add to that instead:

$JsonFile = "build.json"
$JsonObject = $(Get-Content $JsonFile -Raw)
$SystemObject=$($JsonObject | ConvertFrom-Json)
$OldScriptsJson = $SystemObject.provisioners[1].scripts | ConvertTo-Json

Which gives me the old JSON which I want to add to:

[
  "{{template_dir}}/scripts/installers/azcopy.sh",
  "{{template_dir}}/scripts/installers/azure-cli.sh",
  "{{template_dir}}/scripts/installers/azure-devops-cli.sh",
  "{{template_dir}}/scripts/installers/basic.sh",
  "{{template_dir}}/scripts/installers/bicep.sh",
  "{{template_dir}}/scripts/installers/aliyun-cli.sh",
  "{{template_dir}}/scripts/installers/apache.sh",
  "{{template_dir}}/scripts/installers/aws.sh",
  "{{template_dir}}/scripts/installers/clang.sh",
  "{{template_dir}}/scripts/installers/swift.sh",
  "{{template_dir}}/scripts/installers/cmake.sh",
  "{{template_dir}}/scripts/installers/codeql-bundle.sh",
  "{{template_dir}}/scripts/installers/containers.sh",
  "{{template_dir}}/scripts/installers/dotnetcore-sdk.sh",
  "{{template_dir}}/scripts/installers/erlang.sh",
  "{{template_dir}}/scripts/installers/firefox.sh",
  "{{template_dir}}/scripts/installers/gcc.sh",
  "{{template_dir}}/scripts/installers/gfortran.sh",
  "{{template_dir}}/scripts/installers/git.sh",
  "{{template_dir}}/scripts/installers/github-cli.sh",
  "{{template_dir}}/scripts/installers/google-chrome.sh",
  "{{template_dir}}/scripts/installers/google-cloud-sdk.sh",
  "{{template_dir}}/scripts/installers/haskell.sh",
  "{{template_dir}}/scripts/installers/heroku.sh",
  "{{template_dir}}/scripts/installers/hhvm.sh",
  "{{template_dir}}/scripts/installers/java-tools.sh",
  "{{template_dir}}/scripts/installers/kubernetes-tools.sh",
  "{{template_dir}}/scripts/installers/oc.sh",
  "{{template_dir}}/scripts/installers/leiningen.sh",
  "{{template_dir}}/scripts/installers/miniconda.sh",
  "{{template_dir}}/scripts/installers/mono.sh",
  "{{template_dir}}/scripts/installers/kotlin.sh",
  "{{template_dir}}/scripts/installers/mysql.sh",
  "{{template_dir}}/scripts/installers/mssql-cmd-tools.sh",
  "{{template_dir}}/scripts/installers/sqlpackage.sh",
  "{{template_dir}}/scripts/installers/nginx.sh",
  "{{template_dir}}/scripts/installers/nvm.sh",
  "{{template_dir}}/scripts/installers/nodejs.sh",
  "{{template_dir}}/scripts/installers/bazel.sh",
  "{{template_dir}}/scripts/installers/oras-cli.sh",
  "{{template_dir}}/scripts/installers/phantomjs.sh",
  "{{template_dir}}/scripts/installers/php.sh",
  "{{template_dir}}/scripts/installers/postgresql.sh",
  "{{template_dir}}/scripts/installers/pulumi.sh",
  "{{template_dir}}/scripts/installers/ruby.sh",
  "{{template_dir}}/scripts/installers/r.sh",
  "{{template_dir}}/scripts/installers/rust.sh",
  "{{template_dir}}/scripts/installers/julia.sh",
  "{{template_dir}}/scripts/installers/sbt.sh",
  "{{template_dir}}/scripts/installers/selenium.sh",
  "{{template_dir}}/scripts/installers/smallstep-cli.sh",
  "{{template_dir}}/scripts/installers/terraform.sh",
  "{{template_dir}}/scripts/installers/packer.sh",
  "{{template_dir}}/scripts/installers/vcpkg.sh",
  "{{template_dir}}/scripts/installers/dpkg-config.sh",
  "{{template_dir}}/scripts/installers/mongodb.sh",
  "{{template_dir}}/scripts/installers/android.sh",
  "{{template_dir}}/scripts/installers/yq.sh",
  "{{template_dir}}/scripts/installers/pypy.sh",
  "{{template_dir}}/scripts/installers/python.sh",
  "{{template_dir}}/scripts/installers/graalvm.sh"
]

What I want to do

I basically want to add the following JSON to the old JSON

[
    "{{template_dir}}/scripts/installers/colordiff.sh",
    "{{template_dir}}/scripts/installers/kubelogin.sh",
    "{{template_dir}}/scripts/installers/kubeval.sh",
    "{{template_dir}}/scripts/installers/kubeaudit.sh",
    "{{template_dir}}/scripts/installers/smallstep-cli.sh"
  ]

So the final product of all of this can be saved back to the original JSON:

{
  "builders": [
    {
      "type": "azure-arm"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/docker-compose.sh",
        "{{template_dir}}/scripts/installers/docker-moby.sh"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}",
        "DOCKERHUB_LOGIN\u003d{{user `dockerhub_login`}}",
        "DOCKERHUB_PASSWORD\u003d{{user `dockerhub_password`}}"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} {{ .Path }}\u0027"
    },
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/azcopy.sh",
        "{{template_dir}}/scripts/installers/azure-cli.sh",
        "{{template_dir}}/scripts/installers/azure-devops-cli.sh",
        "{{template_dir}}/scripts/installers/basic.sh",
        "{{template_dir}}/scripts/installers/bicep.sh",
        "{{template_dir}}/scripts/installers/aliyun-cli.sh",
        "{{template_dir}}/scripts/installers/apache.sh",
        "{{template_dir}}/scripts/installers/aws.sh",
        "{{template_dir}}/scripts/installers/clang.sh",
        "{{template_dir}}/scripts/installers/swift.sh",
        "{{template_dir}}/scripts/installers/cmake.sh",
        "{{template_dir}}/scripts/installers/codeql-bundle.sh",
        "{{template_dir}}/scripts/installers/containers.sh",
        "{{template_dir}}/scripts/installers/dotnetcore-sdk.sh",
        "{{template_dir}}/scripts/installers/erlang.sh",
        "{{template_dir}}/scripts/installers/firefox.sh",
        "{{template_dir}}/scripts/installers/gcc.sh",
        "{{template_dir}}/scripts/installers/gfortran.sh",
        "{{template_dir}}/scripts/installers/git.sh",
        "{{template_dir}}/scripts/installers/github-cli.sh",
        "{{template_dir}}/scripts/installers/google-chrome.sh",
        "{{template_dir}}/scripts/installers/google-cloud-sdk.sh",
        "{{template_dir}}/scripts/installers/haskell.sh",
        "{{template_dir}}/scripts/installers/heroku.sh",
        "{{template_dir}}/scripts/installers/hhvm.sh",
        "{{template_dir}}/scripts/installers/java-tools.sh",
        "{{template_dir}}/scripts/installers/kubernetes-tools.sh",
        "{{template_dir}}/scripts/installers/oc.sh",
        "{{template_dir}}/scripts/installers/leiningen.sh",
        "{{template_dir}}/scripts/installers/miniconda.sh",
        "{{template_dir}}/scripts/installers/mono.sh",
        "{{template_dir}}/scripts/installers/kotlin.sh",
        "{{template_dir}}/scripts/installers/mysql.sh",
        "{{template_dir}}/scripts/installers/mssql-cmd-tools.sh",
        "{{template_dir}}/scripts/installers/sqlpackage.sh",
        "{{template_dir}}/scripts/installers/nginx.sh",
        "{{template_dir}}/scripts/installers/nvm.sh",
        "{{template_dir}}/scripts/installers/nodejs.sh",
        "{{template_dir}}/scripts/installers/bazel.sh",
        "{{template_dir}}/scripts/installers/oras-cli.sh",
        "{{template_dir}}/scripts/installers/phantomjs.sh",
        "{{template_dir}}/scripts/installers/php.sh",
        "{{template_dir}}/scripts/installers/postgresql.sh",
        "{{template_dir}}/scripts/installers/pulumi.sh",
        "{{template_dir}}/scripts/installers/ruby.sh",
        "{{template_dir}}/scripts/installers/r.sh",
        "{{template_dir}}/scripts/installers/rust.sh",
        "{{template_dir}}/scripts/installers/julia.sh",
        "{{template_dir}}/scripts/installers/sbt.sh",
        "{{template_dir}}/scripts/installers/selenium.sh",
        "{{template_dir}}/scripts/installers/smallstep-cli.sh",
        "{{template_dir}}/scripts/installers/terraform.sh",
        "{{template_dir}}/scripts/installers/packer.sh",
        "{{template_dir}}/scripts/installers/vcpkg.sh",
        "{{template_dir}}/scripts/installers/dpkg-config.sh",
        "{{template_dir}}/scripts/installers/mongodb.sh",
        "{{template_dir}}/scripts/installers/android.sh",
        "{{template_dir}}/scripts/installers/yq.sh",
        "{{template_dir}}/scripts/installers/pypy.sh",
        "{{template_dir}}/scripts/installers/python.sh",
        "{{template_dir}}/scripts/installers/graalvm.sh",
        "{{template_dir}}/scripts/installers/colordiff.sh",
        "{{template_dir}}/scripts/installers/kubelogin.sh",
        "{{template_dir}}/scripts/installers/kubeval.sh",
        "{{template_dir}}/scripts/installers/kubeaudit.sh",
        "{{template_dir}}/scripts/installers/smallstep-cli.sh"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}",
        "DEBIAN_FRONTEND\u003dnoninteractive"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} {{ .Path }}\u0027"
    },
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/Install-Toolset.ps1",
        "{{template_dir}}/scripts/installers/Configure-Toolset.ps1"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} pwsh -f {{ .Path }}\u0027"
    }
  ]
}

How can I go about that using PowerShell?

zett42 · Accepted Answer · 2022-05-09T15:10:06.677

This should do it:

$json = Get-Content 'Build.json' -Raw | ConvertFrom-Json

# Define an array of strings to add.
$scriptsToAdd = @(
    '{{template_dir}}/scripts/installers/colordiff.sh'
    '{{template_dir}}/scripts/installers/kubelogin.sh'
    '{{template_dir}}/scripts/installers/kubeval.sh'
    '{{template_dir}}/scripts/installers/kubeaudit.sh'
    '{{template_dir}}/scripts/installers/smallstep-cli.sh'
)
# Alternatively read these from a JSON file as well:
# $scriptsToAdd = Get-Content 'ScriptsToAdd.json' -Raw | ConvertFrom-Json

# Append the $scriptsToAdd array to the scripts array of the JSON
$json.provisioners[1].scripts += $scriptsToAdd

# Make sure to specify a large enough argument for -Depth, which defaults to 2.
# Otherwise objects that are nested deeper will be truncated.
$json | ConvertTo-Json -Depth 100 | Set-Content 'BuildNew.json'

As for what you have tried:

$OldScriptsJson = $SystemObject.provisioners[1].scripts | ConvertTo-Json

You just converted part of the original JSON object back to a JSON string. When working with JSON, you should always operate on the object and only as a final step, convert back to the serialized JSON, which can be a string or a file.

Works perfect! Thank you. You do not want to know how long I was stuck on this. — craigtho, May 09 '22 at 15:33

js2010 · Answer 2 · 2022-05-09T14:40:44.037

0

$SystemObject = get-content build.json | convertfrom-json

# load changes into another object
$addobject = get-content add.json | convertfrom-json

# add 2 arrays together (+= kills puppies in bulk)
$SystemObject.provisioners[1].scripts += $addobject

# save, watch out for default depth of 2, the latest ps 7 gives a warning 
# "WARNING: Resulting JSON is truncated as serialization has exceeded the set depth
# of 2."
$SystemObject | ConvertTo-Json -Depth 3 | set-content build2.json

edited May 09 '22 at 14:40

answered May 09 '22 at 14:35

js2010

23,033
6
64
66

Nice! We almost got the same solution. Note that `ConvertFrom-Json` may behave strangely when passing multiple strings (lines) as input, so it is [recommended](https://stackoverflow.com/a/71053689/7571258) to always pass `-Raw` to `Get-Content` to pass a single (possibly multiline) string to `ConvertFrom-Json`. – zett42 May 09 '22 at 15:01

Convert JSON to a PowerShell Object, then, replace that Object with a new Object and convert back to JSON

Background

What I have tried

What I want to do

2 Answers2

Linked