jq: create array of object in json and insert the new object each time bash scripts executes

Question

I want to create valid json using jq in bash. each time when bash script will execute "Add new element to existing JSON array" and if file is empty create new file.

I am using following jq command to create my json (which is incomplete, please help me to complete it)

$jq -n -s '{service: $ARGS.named}' \
         --arg transcationId $TRANSACTION_ID_METRIC '{"transcationId":"\($transcationId)"}' \
         --arg name $REALPBPODDEFNAME '{"name ":"\($name )"}'\
         --arg lintruntime $Cloudlintruntime '{"lintruntime":"\($lintruntime)"}' \
         --arg status $EXITCODE '{"status":"\($status)"}' \
         --arg buildtime $totaltime '{"buildtime":"\($buildtime)"}' >> Test.json

which is producing output like


{
  "service": {
    "transcationId": "12345",
    "name": "sdsjkdjsk",
    "lintruntime": "09",   
    "status": "0",
    "buildtime": "9876"
  }
}
{
  "service": {
    "transcationId": "123457",
    "servicename": "sdsjkdjsk",
    "lintruntime": "09",   
    "status": "0",
    "buildtime": "9877"
  }
}

but I don't want output in this format

json should be created first time like

what should be jq command for creating below jason

{ 
   "ServiceData":{ 
      "date":"30/1/2020",
      "ServiceInfo":[ 
         { 
            "transcationId":"20200129T130718Z",
            "name":"MyService",
            "lintruntime":"178",
            "status":"0",
            "buildtime":"3298"
         }
      ]
   }
}

and when I next time execute the bash script element should be added into the array like

what is the jq command for getting json in this format

{ 
   "ServiceData":{ 
      "date":"30/1/2020",
      "ServiceInfo":[ 
         { 
            "transcationId":"20200129T130718Z",
            "name":"MyService",
            "lintruntime":"16",
            "status":"0",
            "buildtime":"3256"
         },
         { 
            "transcationId":"20200129T130717Z",
            "name":"MyService",
            "lintruntime":"16",
            "status":"0",
            "buildtime":"3256"
         }
      ]
   }
}

also I want "date " , "service data" , "service info"
fields in my json which are missing in my current one

Does this answer your question? [Add new element to existing JSON array with jq](https://stackoverflow.com/questions/42245288/add-new-element-to-existing-json-array-with-jq) — lojza, Jan 30 '20 at 14:55
lojza.. I can achieve it insertion of one element at single time ... but I my scenario same file should be modified each time with the insertion .. how that can be achieved modification of same file — XOR, Jan 31 '20 at 07:08
Check https://stackoverflow.com/questions/36565295/jq-to-replace-text-directly-on-file-like-sed-i — lojza, Jan 31 '20 at 10:17

chepner · Answer 1 · 2020-06-05T13:06:29.270

1

You don't give a separate filter to each --arg option; it just defines a variable which can be used in the single filter argument. You just want to add new object to your input. jq doesn't do in-place file editing, so you'll have to write to a temporary file and replace your original after the fact.

jq --arg transactionId "$TRANSACTION_ID_METRIC" \
   --arg name "$REALPBPODDEFNAME" \
   --arg lintruntime "$Cloudlintruntime" \
   --arg status "$EXITCODE" \
   --arg buildtime "$totaltime" \
   '.ServiceData.ServiceInfo += [ {transactionID: $transactionId,
                                   name: $name,
                                   lintruntime: $lintruntime,
                                   status: $status,
                                   buildtime: $buildtime
                                   }]' \
    Test.json > tmp.json &&
 mv tmp.json Test.json

Here's the same command, but using an array to store all the --arg options and a variable to store the filter so the command line is a little simpler. (You also don't need explicit line continuations inside an array definition.)

args=(
   --arg transactionId "$TRANSACTION_ID_METRIC"
   --arg name "$REALPBPODDEFNAME"
   --arg lintruntime "$Cloudlintruntime"
   --arg status "$EXITCODE"
   --arg buildtime "$totaltime"
)

filter='.ServiceData.ServiceInfo += [
    {
      transactionID: $transactionId,
      name: $name,
      lintruntime: $lintruntime,
      status: $status,
      buildtime: $buildtime
    }
]'

jq "${args[@]}" "$filter" Test.json > tmp.json && mv tmp.json Test.json

edited Jun 05 '20 at 13:06

answered Jan 30 '20 at 16:09

chepner

497,756
71
530
681

somehow above code is not working gives .......................... error jq: error: $transactionId is not defined at , line 1: .ServiceData.ServiceInfo += [ {transactionId: $transactionId, jq: 1 compile error – XOR Jan 31 '20 at 05:57
I completely new to bash scripting .. so no idea how to solve it – XOR Jan 31 '20 at 05:59
my command is .................cat metricData.json | $jq --arg transcationId "${TRANSACTION_ID_METRIC}" \ '.ServiceData.ServiceInfo += [{"transcationID": $transactionId}]' >tmp.json && mv tmp.json metricData.json ............ and it's giving issue ........................jq: error: $transactionId is not defined at , line 1: .ServiceData.ServiceInfo += [{"transcationID": $transactionId}] – XOR Jan 31 '20 at 10:06
Just a typo on my part; I swapped the `a` and the `c` in `transaction`. – chepner Jan 31 '20 at 12:40
@ chepner I have tried on different linux VM and it's giving tmp.json: Permission denied do you have any idea why I am seeing this – XOR Jun 05 '20 at 12:38
`Test.json` is an argument to `jq`, not a command. Did you forget the backslash on the preceding line? (Or add additional whitespace after the backslash so that it no longer acted as a line continuation character?) I added some indentation to emphasize that. – chepner Jun 05 '20 at 13:06
@ chepner does that tmp.json gets created in current working directory ? and will gets deleted at the end? – XOR Jun 05 '20 at 14:13
Not deleted, but renamed. `mv tmp.json Test.json` is atomic, so it has exactly one of two outcomes: `mv` works, and `tmp.json` replaces `Test.json`, or `mv` doesn't work, leaving both `tmp.json` and `Test.json` unchanged. – chepner Jun 05 '20 at 14:36

jq: create array of object in json and insert the new object each time bash scripts executes

1 Answers1