splitting json into multiple files in windows

Question

I am trying to split my json object into multiple json arrays.

{
    "MusicCollection": [
        {
            "PutRequest": {
                "Item": {
                    "Artist": {"S": "No One You Know"},
                    "SongTitle": {"S": "Call Me Today"},
                    "AlbumTitle": {"S": "Somewhat Famous"}
                }
            }
        },
        {
            "PutRequest": {
                "Item": {
                    "Artist": {"S": "Acme Band"},
                    "SongTitle": {"S": "Happy Day"},
                    "AlbumTitle": {"S": "Songs About Life"}
                }
            }
        },
        {
            "PutRequest": {
                "Item": {
                    "Artist": {"S": "No One You Know"},
                    "SongTitle": {"S": "Scared of My Shadow"},
                    "AlbumTitle": {"S": "Blue Sky Blues"}
                }
            }
        }
    ]
}

Command in below post works in linux for splitting the file into multiple putRequest arrays.

jq -cM --argjson sublen '2' 'range(0; length; $sublen) as $i | .[$i:$i+$sublen]' \
input.json | split -l 1 -da 3 - meta2_

Reference: Split a JSON array into multiple files using command line tools

Directly running below command in command prompt gives me 3 put requests.

jq ".MusicCollection[]" music.json

However, when I used below for loop in batch script, it just gives me }.

    for /f "tokens=*" %%a in ('music.json') do (
        jq ".MusicCollection[]" music.json
        set array_value=%%a
    )

echo Array: %array_value%

Can someone please guide me as how I should get the same output of command prompt with the for loop?

Answer 1

Since you have jq in your environment, it seems reasonable to assume that you could also use awk without too much difficulty. If so, you could pipe the JSONLines output of jq (invoked with the -c option) to an awk command along the following lines:

awk -v mx=2 'BEGIN{n=0}
     n%mx == 0 {
       nf++; close(out);
       out="OUTPUT."nf".json";
     }
     {n++; print > out}' 

Answer 2

This one allows you to split putRequest to set of arrays having no more than 2 items

jq -c "[ .MusicCollection[] ] | range(0; length; 2) as $i | { MusicCollection: .[ $i:$i+2 ] }" input.json

I guess this could help you to understand the following script. It assumes that the names of each file generated match the pattern FileN (where N is a number).

@echo off

rem stop variable expansion while using them under loop
setlocal enabledelayedexpansion

rem initialize the counters
set /a "FILENUM=0"
set /a "ARRAYLENGTH=2"

for /f "tokens=*" %%a in ( '
    jq -c --argjson L "!ARRAYLENGTH!" "[ .MusicCollection[] ] | range(0; length; $L) as $i | { MusicCollection: .[ $i:$i+$L ] }" input.json
' ) do (
    rem set the counter to the next file
    set /a "FILENUM+=1"

    rem print results
    echo:File !FILENUM!:
    echo:%%~a
    echo:

    rem uncomment to turn on printing to files called FileN
    rem echo:%%~a>File!FILENUM!
)