0

I have a scenario where I have to split a csv in certain number of rows, and for each batch there should be json file generated through PowerShell Script.

Here is what I am doing as of now:

$csv = "C:\Desktop\report.csv"

[int]$totalRows = 0
$reader = New-Object IO.StreamReader $csv
while($reader.ReadLine() -ne $null) { $totalRows++ }
$reader.Dispose()

$totalRows
$startRow = 0
$counter = 1

while($startRow -lt $totalRows)
{   
    Import-CSV $csv | Select-Object @{expression = { "Append this value"+ $_.Name}; label = 'NewName'}, @{expression = {$_.Account}; label = 'AccountNumber'} | Select-Object -skip $startRow -first 2 | ConvertTo-Json | Add-Content -Path "C:\Desktop\r_$($counter).json"
    
    $startRow += 2
    $counter++
}

The only problem here is that I am not able to enclose the Account number values in square bracket []:

Actual : "AccountNumber": "123" expected : "AccountNumber": ["123"]

Also I am not sure on how to put whole json in each file under a root element through this. Also not sure if this "ConvertTo-Json" is the way to go as csv data needs to be edited, Please help.

Here is a csv for reference-

Name,Account,Role
John,123,Second
Rocky,345,Third
Tony,234,First
Rocky,345,Second
Matt,999,Second
Bernard,888,Third
Matt,999,First
Jacob,789,Second
Angela,777,Second
Jacob,789,First

Expected Output

First File:

{   
    "details":
    [
        {
            "NewName":  "Append this valueJohn",
            "AccountNumber":  ["123","333"]
        },
        {
            "NewName":  "Append this valueRocky",
            "AccountNumber":  ["345"]
        }
    ]
}

Second File:

{   
    "details":
    [
        {
            "NewName":  "Append this valueTony",
            "AccountNumber":  ["234"]
        },
        {
            "NewName":  "Append this valueRocky",
            "AccountNumber":  ["345"]
        }
    ]
}

So on till 6th file:

{   
    "details":
    [
        {
            "NewName":  "Append this valueAngela",
            "AccountNumber":  ["777"]
        },
        {
            "NewName":  "Append this valueJacob",
            "AccountNumber":  ["789"]
        }
    ]
}

Thanks
  • 2
    Why do you need it in square brackets? it is not an array – Mickey Cohen Oct 25 '21 at 08:17
  • Change `expression = {$_.Account}` to `expression = {'[{0}]' -f $_.Account)}` – Mathias R. Jessen Oct 25 '21 at 08:25
  • @MathiasR.Jessen This is writing the output in slight different way than expected: "[123]", I am trying to get it as ["123"], I guess it should be converted to array somehow. – Amit Chouksey Oct 25 '21 at 09:10
  • 1
    @AmitChouksey Are you perhaps mistaking CSV for JSON? `[]` for lists/arrays is not a common practice in CSV generation – Mathias R. Jessen Oct 25 '21 at 09:12
  • `$Csv | Select-Object @{expression = { "Append this value"+ $_.Name}; label = 'NewName'}, @{expression = { ,@([String]$_.Account) }; label = 'AccountNumber'} | Select-Object -skip $startRow -first 2 | ConvertTo-Json -Compress`? – iRon Oct 25 '21 at 09:57
  • Sorry but I didnt understand "lists/arrays is not a common practice in CSV generation". @iRon: this is throwing this: "AccountNumber": { "value": [ "123,333" ], "Count": 1 } Which is very different. – Amit Chouksey Oct 25 '21 at 12:09
  • Please add the expected output (of `C:\Desktop\r_$($counter).json`) to the question. Btw, it might look different, but the from a [`json`](https://en.wikipedia.org/wiki/JSON) point of view it is still valid and the contents is in fact the same as when you omit the `-Compress` parameter where the lines (including the square brackets) are placed on different lines). – iRon Oct 25 '21 at 12:46
  • expected output: { "details": [ { "NewName": "Append this valueJohn", "AccountNumber": ["123","333"] #for more than one values }, { "NewName": "Append this valueRocky", "AccountNumber": ["345"] #for single values } ] } – Amit Chouksey Oct 25 '21 at 12:53
  • So, why are you using `Json` ([ConvertTo-Json](https://docs.microsoft.com/powershell/module/microsoft.powershell.utility/convertto-json))? Square brackets have a special meaning in [`Json`](https://en.wikipedia.org/wiki/JSON) (quote: "*Arrays use square bracket notation...*") and therefore automatically escaped with: `\[` when converted from an object. If you expect `"AccountNumber": ["123"]`, why not simply: `|Select-Object @{expression = { "Append this value"+ $_.Name}; label = 'NewName'}, @{expression = { """AccountNumber"": [""$($_.Account)""]"}; label = 'AccountNumber'}`? Why the `Json`? – iRon Oct 25 '21 at 13:11
  • @iRon its just that I will have to ask the other team who is expecting this input, I am not sure if [] can be omitted, but last I asked they were keen on this. And I am very new to PowerShell, I still dont understand the expressions, I am trying to understand what our comment meant and explore more but no luck :( Edit: Just wanted to add, if you look my whole requirement it is to break csv into batches and then convert to json. The code I am using is doing that but I doubt if its a good idea to use convertTo-Json – Amit Chouksey Oct 25 '21 at 13:18
  • Also note that you can't just append to `Json` like `ConvertTo-Json |Add-Content -Append` for the same reason. It will invalidate your `Json` as arrays (multiple items) need to be surrounded by square brackets. It is still not clear what you expect a `Json` file or a `csv` file (I guess a `Json` file but that will not shown from a single property/value). Please add the whole expected contents (of a single iteration) to the question. I think you will need to investigate in what `Json` is to be able to satisfy "the other team" with a correct file. – iRon Oct 25 '21 at 13:26
  • just added expected output iteration wise – Amit Chouksey Oct 25 '21 at 13:35
  • Ok, it is clear now that you expecting an `Json` file but the logic behind the contents is still unclear to fully answer the question: why are there two values (`["123","333"]`) for `John`? Why are there just two accounts in a file? Anyways, as commented earlier: use the [Array subexpression operator](https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_operators) to force an array which will surround the value in brackets in your json result. See also: [Add square brackets while convert it to json in powershell](https://stackoverflow.com/a/52584436/1701026) – iRon Oct 25 '21 at 14:09

1 Answers1

0

Continuing from my comments as I can't use newlines in there:

Converting the below PowerShell object to Json (JavaScript Object Notation):

@{
    details = @{
        NewName = 'Append this valueJohn'
        AccountNumber = 123
    }, @{
        NewName = 'Append this valueRocky'
        AccountNumber = 345
    }
} |ConvertTo-Json

Results in

{
  "details": [
    {
      "NewName": "Append this valueJohn",
      "AccountNumber": 123
    },
    {
      "NewName": "Append this valueRocky",
      "AccountNumber": 345
    }
  ]
}

If you want to quote the values, you will need to force them to a string
(by using quotes or e.g. the [String] initializer):

@{
    details = @{
        NewName = 'Append this valueJohn'
        AccountNumber = "123"
    }, @{
        NewName = 'Append this valueRocky'
        AccountNumber = [String]345
    }
} |ConvertTo-Json

Results:

{
  "details": [
    {
      "NewName": "Append this valueJohn",
      "AccountNumber": "123"
    },
    {
      "NewName": "Append this valueRocky",
      "AccountNumber": "345"
    }
  ]
}

If you also want to put the single item (scalar) strings between square brackets (which represents an array), you will need to force the values to an array using the Array subexpression operator @( ) or the Comma operator ,. Note that this automatically happens when it concerns multiple values, e.g.: AccountNumber = "123", "333"

@{
    details = @{
        NewName = 'Append this valueJohn'
        AccountNumber = @("123")
    }, @{
        NewName = 'Append this valueRocky'
        AccountNumber = ,[String]345
    }
} |ConvertTo-Json -Depth 9

(Also note the -Depth parameter)
Results:

{
  "details": [
    {
      "NewName": "Append this valueJohn",
      "AccountNumber": [
        "123"
      ]
    },
    {
      "NewName": "Append this valueRocky",
      "AccountNumber": [
        "345"
      ]
    }
  ]
}

Note that the square brackets aren't on the same line as per default the ConvertTo-Json expands the output. You might consider to use the -Compress parameters but that will put everything on the same line:

{"details":[{"NewName":"Append this valueJohn","AccountNumber":["123"]},{"NewName":"Append this valueRocky","AccountNumber":["345"]}]}

In other words, even the appearance is different, it all technically equal and represents the same object.

iRon
  • 20,463
  • 10
  • 53
  • 79
  • So I have finally used the preparing a custom json approach, and it worked, though I am not convinced that it is the best way but here is a summary: first : $accountNumber = @($line.Account) -replace ',', '","' then in script $jsonData = @" { .... $AccountNumber = ["$accountNumber"] } there were other edits too such as seperating the nested json with comma, but I was able to use replace and regex to get it done. – Amit Chouksey Oct 26 '21 at 07:15