Using IFS to split a String into an Array with delimiter in bash

Question

I have a variable data="password: password@123, url: url@123, username: user@123". I am trying to split this using the delimiter comma(,). I came up with the following script and it works well in CentOS Linux. But the same does not work in Debian GNU/Linux 9 (Stretch).

IFS=, read -a array <<< $data
echo ${array[1]}
echo ${array[2]}
echo ${array[3]}

Gives the output (CentOS Linux):

password: password@123
url: url@123
username: user@123

But for Debian GNU/Linux 9 (Stretch) the output is empty. Kindly help me with it!

EDIT: The value for data is obtained from a JSON response.

data=$(vault kv get -format=json myapp/database | jq '.data' | sed '0,/{/ s/{//' | sed -r 's/(.*)}/\1 /' | sed s/\"//g)
echo $data
password: password@123, url: url@123, username: user@123

The response of vault kv get -format=json myapp/database:

{
  "request_id": "5eee87f9-16dc-fd62-e249-8737a6cd7041",
  "lease_id": "",
  "lease_duration": 2764800,
  "renewable": false,
  "data": {
    "password": "password@123",
    "url": "url@123",
    "username": "user@123"
  },
  "warnings": null
}

SOLUTION: This script works in both Centos and Ubuntu

data=$(vault kv get -format=json myapp/database | jq '.data')
data=$(echo $data | sed 's/ //g' | sed '0,/{/ s/{//' | sed -r 's/(.*)}/\1 /' | sed 's/\"//g')
IFS=, read -a array <<< $data

Try quoting your variables, `data="password: password@123, url: url@123, username: user@123"; IFS=, read -a array <<< "$data"; declare -p array` Or `printf '%s\n' "${array[0]}" "${array[1]}" "${array[2]}"` — Jetchisel, May 11 '21 at 04:40
Also https://www.shellcheck.net is a good resource for shell scripts. — Jetchisel, May 11 '21 at 04:41
I can't repro; I get the same result on both platforms (CentOS 8.3.2011); the array element `${array[2]}` is empty, but that's because you are using the wrong indices (the first element is in `${array[0]}`). See https://pastebin.com/KyDSCfYk — tripleee, May 11 '21 at 04:50
If that's not it, could you still [edit] to include the output from running the script with `bash -x` on Debian? — tripleee, May 11 '21 at 05:23
As an aside, `jq -r ' .data | to_entries[] | "\(.key): \(.value)"'` avoids the fugly `sed` postprocessing. I doubt you really particularly want to split on commas; this produces newline-delimited output. — tripleee, May 11 '21 at 05:39
@tripleee The script works fine in CentOS but not on Ubuntu Debian. Any thought on it? — AkshayBadri, May 11 '21 at 06:09
You mean other than [what I already suggested?](https://stackoverflow.com/questions/67480611/using-ifs-to-split-a-string-into-an-array-with-delimiter-in-bash?noredirect=1#comment119273564_67480611) — tripleee, May 11 '21 at 06:14
Put a newline at the end of ``$data``. `read -a array <<< "$data"$'\n'`. — Darkman, May 11 '21 at 09:32
@tripleee At least not for my ``read``. Without a newline character, ``read`` seems to ignore the line. — Darkman, May 11 '21 at 09:52
@B.Akshay Instead of using ``read``, why not just using **for loop**. `IFS=, local i=1; for what in $data; do case $i in 1) echo '1='"$what" ;; 2) echo '2='"$what" ;; esac; i=$((i + 1)); done` — Darkman, May 11 '21 at 10:05
... or then hack the `jq` output to provide it in a more useful format. With Bash 5 you could have it print the assignment syntax for an associative array. — tripleee, May 11 '21 at 10:15
@Darkman No repro; I wonder how exactly you are testing that? https://ideone.com/graOFa — tripleee, May 11 '21 at 10:17
You really should combine your multiple `sed` invocations. `sed` is a scripting language; it is not limited to a single command per invocation. (But then even better would be to clean up your `jq` so you don't need `sed` at all.) — tripleee, May 11 '21 at 11:42

score 0 · Answer 1 · answered May 12 '21 at 09:33

The code in your question does not reproduce the problem. The question was rightfully closed, and we still are not sure why the code you posted was not working.

However, the proper fix is probably to refactor your jq command line to output the results you want, instead of try to fix it up with a sequence of messy sed commands.

#!/bin/bash
vault kv get -format=json myapp/database |
jq -r '.data | to_entries[] | "\(.key) \(.value)"')" |
while IFS= read -r key value; do
    printf -v "$key" %s "$value"
done

Perhaps a better design is to read these into an associative array, provided your Bash is new enough (5+).

#!/bin/bash
declare -A config
vault kv get -format=json myapp/database |
jq -r '.data | to_entries[] | "\(.key) \(.value)"')" |
while IFS= read -r key value; do
    config["$key"]="$value"
done

Perhaps there are ways to get vault kv get to format its output in a format which is more suitable for using in a shell script, too.

(Tangentially, nearly anything which looks like sed 's/a/b/' | sed 's/x/y/g' is better refactored into sed 's/a/b/;s/x/y/g'. See also Combining two sed commands)

https://www.vaultproject.io/docs/commands/kv/get suggests that its default output format is probably more suitable for what you are trying to do. JSON is fine if your consumer understands JSON, but Bash doesn't, so you are just complicating things. — tripleee, May 12 '21 at 09:37

Using IFS to split a String into an Array with delimiter in bash

1 Answers1