0

In my shell script I am making an API call using cURL which is stored in variable named token_response, further I need to parse out value of key aws_access_key_id for which I am using sed as shown below. Wondering what's wrong with the pattern in sed which is not able to parse out desired key's value.

When trying to fetch using -

echo aws_access_key_id is:$(echo $token_response | sed -n 's/^.*"AccessKeyId":"\([^"]*\)",*$/\1/p')

JSON is stored in variable called - token_response

    {  
   "Code":"Success",
   "LastUpdated":"2018-12-27T07:16:31Z",
   "Type":"fakedTypeValue",
   "AccessKeyId":"fakedAccessKeyIdValue",
   "Token":"fakedTokenValue"
    }

Following is printed on console without the token value (Seems value for AccessKeyId is not parsed) - aws_access_key_id is:

I expected following - aws_access_key_id is:fakedAccessKeyIdValue

inCode
  • 103
  • 1
  • 10
  • 5
    Better to use `jq` instead of `sed` – anubhava Dec 27 '18 at 17:21
  • 3
    `jq -r '.AccessKeyId' <<< "$token_response"` – anubhava Dec 27 '18 at 17:23
  • 3
    Strongly agree with @anubhava. The short answer to your question is "no, unless the json is formatted just right". JSON structure does not lend itself to regular expressions. It's best not to make assumptions about newlines in json. Actually, aws cli commands can run jq-like result filter for you so you just get the value you want. If you don't need the other fields from `$token_response`, and you include the command that generated that data, I'll show you how to tweak it so that you just get the access key. This may or may not be appopriate for your use case. – erik258 Dec 27 '18 at 17:25
  • @anubhava - Thanks for suggesting, problem is server where above has to happen does not have access to install jq by running `sudo apt-get install jq`. Else I would have used actual parsing solutions instead of sed. – inCode Dec 27 '18 at 17:26
  • 5
    [Should I use sed to parse json](https://www.google.com/search?client=firefox-b-1-ab&q=should+I+use+sed+to+parse+json) leads you to [Parsing JSON with Unix tools](https://stackoverflow.com/q/1955505/608639). Possible duplicate of [Parsing JSON with Unix tools](https://stackoverflow.com/q/1955505/608639) and [Read the json data in shell script](https://stackoverflow.com/q/20488315/608639). – jww Dec 27 '18 at 17:28
  • 1
    As I expected, `jq` is not pre-installed so I am getting `s3-bydate.sh: line 25: jq: command not found` My line 25 is like this - `echo aws_access_key_id is:$(jq -r '.AccessKeyId' <<< "$token_response")`. Let me work with network architects so I can install `jq`. I don't like workarounds (sed in this case) myself and would rather use most efficient and reliable solution (`jq` as suggested above). Thanks everyone. Will update this thread on any new findings. – inCode Dec 27 '18 at 17:44
  • 1
    @inCode: Try this `grep -oP '"AccessKeyId"\h*:\h*"\K[^"]+' <<< "$token_response"` – anubhava Dec 27 '18 at 17:50
  • @anubhava - It works, if you post this as an answer I will accept. – inCode Dec 27 '18 at 18:11
  • I am still going to chase down network engineers to get `jq` installed which is now my preference. For other folks on StackOverflow who are not fortunate enough to get `jq` or similar utility @anubhava your answer can help them. – inCode Dec 27 '18 at 18:13

1 Answers1

1

In the absence of jq, you may use this gnu grep:

grep -oP '"AccessKeyId"\h*:\h*"\K[^"]+' <<< "$token_response" 


fakedAccessKeyIdValue

Or even this when token value appears after line breaks:

grep -zoP '"AccessKeyId"\s*:\s*"\K[^"]+' <<< "$token_response"
anubhava
  • 761,203
  • 64
  • 569
  • 643