Pop last string from array with jq and return whole json object

Question

test.json

{
    "Version": "2012-**-**",
    "Statement": [
        { "Effect": "**",
          "Principal": "**",
          "Action": "**",
          "Resource": "***",
          "Condition": {
            "IpAddress": {
                "aws:SourceIp": [ "127.0.0.1", "1.0.0.2" ]
            }
          }
        }
    ]
}

I would like to pop the array of aws:SourceIp so that the 1.0.0.2, or whatever the last element of the array is removed. The return value should be the whole json object with the array missing the 1.0.0.2.

I have come close with this:

echo $(jq '.Statement[0] .Condition.IpAddress."aws:SourceIp" | .[0:-1] ' test.json ) > test.json

With the help of: https://github.com/stedolan/jq/issues/226

This will pop the last element of the array but return only [ "127.0.0.1"] and then push this in to test.json.

I would like it to return the whole json modified so that I can then be pushed in to test.JSON

@Charles Duffy I know this is different from https://stackoverflow.com/questions/38860529/create-json-using-jq-from-pipe-separated-keys-and-values-in-bash/38862221?noredirect=1#comment84129911_38862221 but it is jq related. — user3738936, Feb 02 '18 at 17:38
more ideas https://stackoverflow.com/questions/29772676/update-one-value-in-array-of-dicts-using-jq — LMC, Feb 02 '18 at 17:41
Our markdown is not GitHub-flavored. Triple-quotes do not make a multi-line code block here -- notice how the JSON wasn't syntax-highlighted until I moved into four-space indents. — Charles Duffy, Feb 02 '18 at 17:45
btw, if you *are* going to use a command substitution to rewrite a file in-place (which is dangerous -- it can delete the file if your command fails), quote it. `echo "$(...)"`, not `echo $(...)`. — Charles Duffy, Feb 02 '18 at 17:46
...and even that isn't completely safe, because `echo "$foo" >bar` first opens `bar` with `O_TRUNC`, and then does a bunch of `write()` syscalls. If you get a failure after the `O_TRUNC` but before all the writes are done, you leave behind a file with partial contents (and anything reading it in that interval likewise won't see a fully complete file). — Charles Duffy, Feb 02 '18 at 17:51
regarding the writing of the file, what would you recommend? — user3738936, Feb 02 '18 at 17:54
Demonstrating that in my answer. Basically, write to a different file, and rename it to the destination name only when the write is complete and was successful. This is standard practice -- it's how tools like `sed -i` work under the hood, for example. — Charles Duffy, Feb 02 '18 at 18:07

score 2 · Accepted Answer · answered Feb 02 '18 at 17:47

Use |= to modify the piece in-place but return the larger document.

tempfile=$(mktemp test.json.XXXXXX)
trap 'rm -f -- "$tempfile"' EXIT

if jq '.Statement[0].Condition.IpAddress["aws:SourceIp"] |= .[0:-1]' \
        <test.json >"$tempfile"; then
    chmod --reference=test.json -- "$tempfile" # this is a GNUism; tweak elsewhere.
    mv -- "$tempfile" test.json
else
    rm -f -- "$tempfile"
fi

This version also takes more care to replace the file with an updated one in a way that never can leave partially-written contents.

score 0 · Answer 2 · answered Feb 02 '18 at 22:28

This is basically just a slightly simpler variant of @CharleDuffy's excellent answer, but it does use sponge (part of moreutils), the point being that sponge preserves the permissions of the specified file:

jq '.Statement[0].Condition.IpAddress["aws:SourceIp"] |= .[:-1]' test.json |
  sponge test.json

Pop last string from array with jq and return whole json object

2 Answers2