Powershell - ConvertTo-Json hangs script

Question

Evening all, I have a script block below to add a user to an application in OKTA. It runs perfectly if I define a user ($login) and run the script, however if I wrap the script block below in a ForEach loop it hangs indefinitely without proceeding past the $jsonBody = $body | ConvertTo-Json -compress -depth 10 line. I came to this conclusion by adding numbered Write-Host lines after each command. I have tried removing the -Compress and -depth switches which results in an Error 400 bad request meaning the body for Invoke-Webrequest is badly formed. Can you see any reason for this hanging?

$body = @()
$userRequest = "https://$org/api/v1/users?q=$login"
$WebResponse = Invoke-WebRequest -Headers $headers -Method Get -Uri $userRequest
$user = $webresponse | ConvertFrom-Json
$UserID = $user.id
$Body = @{
            id = $UserID
            scope = "USER"
            credentials = @{
            userName = $login
            }
         }
$jsonBody = $body | ConvertTo-Json -compress -depth 10
$UpdateRequest = "https://$org/api/v1/apps/$AppID/users"
Invoke-WebRequest -Headers $headers -Body $jsonBody -Method POST -Uri $UpdateRequest
Write-Host "$login added to $AppName successfully" -ForegroundColor Green  

ForEach loop example below with method to check where it failed:

$logins = Get-Content "C:\ScriptRepository\Users.txt"

ForEach ($login in $logins) {
    $body = @()
    Write-Host "1" -ForegroundColor Green
    $userRequest = "https://$org/api/v1/users?q=$login"
    Write-Host "2" -ForegroundColor Green
    $WebResponse = Invoke-WebRequest -Headers $headers -Method Get -Uri $userRequest
    Write-Host "3" -ForegroundColor Green
    $user = $webresponse | ConvertFrom-Json
    Write-Host "4" -ForegroundColor Green
    $UserID = $user.id
    Write-Host "5" -ForegroundColor Green
    $Body = @{
                id = $UserID
                scope = "USER"
                credentials = @{
                userName = $login
                }
             }
    Write-Host "6" -ForegroundColor Green
    $jsonBody = $body | ConvertTo-Json -Compress -Depth 10
    Write-Host "7" -ForegroundColor Green
    $UpdateRequest = "https://$org/api/v1/apps/$AppID/users"
    Write-Host "8" -ForegroundColor Green
    Invoke-WebRequest -Headers $headers -Body $jsonBody -Method POST -Uri $UpdateRequest
    Write-Host "$login added to $AppName successfully" -ForegroundColor Green          
}

Below is the content of $jsonBody if I remove -compress -depth 10 from ConvertTo-Json, I have anonymised the email:

{
    "scope":  "USER",
    "credentials":  {
                        "userName":  {
                                         "value":  "user.email@add.com",
                                         "PSPath":  "C:\\ScriptRepository\\Users.txt",
                                         "PSParentPath":  "C:\\ScriptRepository",
                                         "PSChildName":  "Users.txt",
                                         "PSDrive":  "C",
                                         "PSProvider":  "Microsoft.PowerShell.Core\\FileSystem",
                                         "ReadCount":  2
                                     }
                    },
    "id":  "00u1230u44rbpE4z70i7"
}

Answer 1

To add some background information to your own effective solution:

• In Windows PowerShell, ConvertTo-Json doesn't serialize strings as expected if they are decorated with ETS properties, as happens when text files are read with Get-Content.

  • Instead of serializing decorated strings as just string (e.g., "foo", they are serialized as custom objects whose .value property contains the original string, alongside the ETS properties.

  • The objects in these ETS properties, which contain metadata about the file each string was read from, are deeply nested and not designed for JSON serialization. To prevent "runaway" serialization, PowerShell limits the object-graph recursion depth to 2 levels by default. Your use of -Depth 10 caused such runaway serialization, which either takes an excessive amount and memory to complete, or, with circular references in the object graph, keeps running until it runs out of memory.

  • As an aside: While the default serialization depth of 2 happens to be useful for objects not designed for JSON serialization (to minimize the risk of runaway serialization), it is a perennial pitfall when using objects that are, for which you do not want a fixed depth to be enforced, as it results in data loss by truncation. In Windows PowerShell, this truncation is, unfortunately, quiet, whereas in PowerShell (Core) 7.1+ you now at least get a warning - see this answer for background information.

• To serialize such decorated strings as just strings, simply call their .ToString() method to get their undecorated value (alternatively, but more obscurely, access their .psobject.BaseObject property, using the intrinsic psobject property).

  • See below for a more efficient alternative that eliminates the decorations at the Get-Content level.

This problem no longer occurs in PowerShell (Core) 7.1+, where decorated strings ([string]) and date values ([datetime]) now serialize just like undecorated ones; however, instances of all other types that have ETS properties are still serialized as custom objects, as described above - see GitHub issue #5797 for the discussion that led to this change.

---

Efficient alternative to individual .ToString() calls on the lines returned by Get-Content:

An efficient and conceptually simply solution is one of the ones Santiago Squarzon mentioned in a comment:

# Note the -ReadCount 0
# Each value of $login in the loop is then undecorated.
foreach ($login in (Get-Content -ReadCount 0 "C:\ScriptRepository\Users.txt")) {
  # ...
}

• -ReadCount 0 reads all lines at once into a single array, rather than streaming the lines, one by one, only to be collected when captured in a variable or used in an expression.

• Aside from being much faster than letting the lines stream, it is only the array as a whole being returned that gets decorated with the ETS properties, whereas its elements (the strings representing the lines) remain undecorated.

Answer 2

Ok so finally sorted by using this :

$logins = Get-Content "C:\ScriptRepository\Users.txt" | %{$_.PSObject.BaseObject}