jq how to retain the parent square bracket while extracting and modifying content in json

Question

I have the below JSON where I need modify few elements values and also extract only the contents inside containerDefinitions while retaining the square bracket.

The below script works in replacing the required value and extract the json inside the square brack inside containerDefinitions but leaves out the square bracket. Is there additinal command that can retain the square brackets.

echo $TASK_DEFINITION | jq '.taskDefinition.containerDefinitions[0] | ( .environment[] |= if   .name == "SMT_PORT_3306_TCP_ADDR" then .value = "myvalue" elif .name == "SERVER_API_TIMEOUT_SUBSCRIPTIONS_CANCEL_REQUEST" then .value = "myvalue" else . end) | .logConfiguration.options."awslogs-group" = "myvalue" ' > ${TASK_DEFINITION_PATH}/${SERVICE_NAME}2-task-definition.json

Input JSON

{
   "taskDefinition":{
      "taskDefinitionArn":"some value",
      "containerDefinitions":[
         {
            "name":"common-api-img",
            "environment":[
               {
                  "name":"SERVER_API_TIMEOUT_SUBSCRIPTIONS_CANCEL_REQUEST",
                  "value":"false"
               },
               {
                  "name":"SMT_PORT_3306_TCP_ADDR",
                  "value":"valueToReplace"
               }
            ],
            "mountPoints":[
               
            ],
            "volumesFrom":[
               
            ]
         }
      ],
      "revision":65,
      "volumes":[
         
      ],
      "status":"ACTIVE"
   }
}

expected output

[
   {
      "name":"common-api-img",
      "environment":[
         {
            "name":"SERVER_API_TIMEOUT_SUBSCRIPTIONS_CANCEL_REQUEST",
            "value":"myvalue"
         },
         {
            "name":"SMT_PORT_3306_TCP_ADDR",
            "value":"myvalue"
         }
      ],
      "mountPoints":[
         
      ],
      "volumesFrom":[
         
      ]
   }
]

It would be helpful to also include your _actual output_, not just the expected output, so they can be compared just by reading the question. — Charles Duffy, Jun 13 '21 at 15:24
BTW, if `echo $TASK_DEFINITION` isn't changed to `echo "$TASK_DEFINITION"` _with the quotes_, you get the bugs described in [I just assigned a variable, but `echo $variable` shows something else!](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else) — Charles Duffy, Jun 13 '21 at 15:25
(You might also want to avoid all-caps names for your own variables -- these are defined by POSIX to be used for variables meaningful to the shell and OS itself, whereas POSIX-compliant tools are guaranteed not to have their behavior modified by updates to variables with at least one lower-case character in their names; read https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html keeping in mind that modifying a regular shell variable overwrites any environment variable by the same name). — Charles Duffy, Jun 13 '21 at 15:32
(the above is why it's safe to write `for path in ...` in bash, but not to write `for PATH in ...` -- only all-caps variables having side effects on shell behavior). — Charles Duffy, Jun 13 '21 at 15:42

Charles Duffy · Answer 1 · 2021-06-13T15:41:47.170

When you put a [0] into your query, you're throwing away all but the first list entry. Thus, you no longer have a list. Thus, there are no more square brackets.

If you want to process all items, not just the first one (and thus to generate a stream of items), and want to put that stream into a list, you might instead want:

jq '
  .taskDefinition.containerDefinitions | map(
    ( .environment[] |=
      if .name == "SMT_PORT_3306_TCP_ADDR" then
        .value = "myvalue"
      elif .name == "SERVER_API_TIMEOUT_SUBSCRIPTIONS_CANCEL_REQUEST" then
        .value = "myvalue"
      else . end
    ) |
    .logConfiguration.options."awslogs-group" = "myvalue"
  )
'  <<<"$TASK_DEFINITIONS" >"${TASK_DEFINITION_PATH}/${SERVICE_NAME}2-task-definition.json"

Note the removal of [0], and the use of map() to pass all the list elements through the transforming function.

Another approach is to use jq -n, and put inputs inside of a list:

jq -n '
  [ inputs |
    .taskDefinition.containerDefinitions[] |
    ( .environment[] |=
      if .name == "SMT_PORT_3306_TCP_ADDR" then
        .value = "myvalue"
      elif .name == "SERVER_API_TIMEOUT_SUBSCRIPTIONS_CANCEL_REQUEST" then
        .value = "myvalue"
      else . end
    ) |
    .logConfiguration.options."awslogs-group" = "myvalue"
  ]
'  <<<"$TASK_DEFINITIONS" >"${TASK_DEFINITION_PATH}/${SERVICE_NAME}2-task-definition.json"

Note the addition of outer [ and ]s, the use of inputs inside that block, the use of the -n argument to jq, and the use of [] where [0] had been used before.

score 0 · Answer 2 · answered Jun 13 '21 at 18:13

Your jq program adds .logConfiguration whereas the "expected output" does not, but except for that key, the following variation of your jq program retains the [0] and produces the "expected output":

[.taskDefinition.containerDefinitions[0]
 | ( .environment[] |= if .name | IN("SMT_PORT_3306_TCP_ADDR", "SERVER_API_TIMEOUT_SUBSCRIPTIONS_CANCEL_REQUEST")
                       then .value = "myvalue" else . end)
 | .logConfiguration.options."awslogs-group" = "myvalue"
]

Comments:

Notice how the use of IN/1 makes the solution much DRYer.
If your jq does not have IN/1, then now would be a good time to upgrade, but an alternative would be simply to include its def, as shown below.
If you opt for a solution that avoids [0] (e.g. in favor of map), you could still use IN to avoid redundancy.

def IN(s): first((s == .) // empty) // false;

jq how to retain the parent square bracket while extracting and modifying content in json

2 Answers2