Convert multiline string to array

Question

I have this script:

nmapout=`sudo nmap -sP 10.0.0.0/24`
names=`echo "$nmapout" | grep "MAC" | grep -o '(.\+)'`
echo "$names"

now the $names variable contains strings delimited with newlines:

>_
 (Netgear)
 (Hon Hai Precision Ind. Co.)
 (Apple)

I tried to do the array conversion with the sub-string approach:

names=(${names//\\n/ })
echo "${names[@]}"

But the problem is that I can't access them by indexing (i.e., ${names[$i] etc.), if I run this loop

for (( i=0; i<${#names[@]}; i++ ))
do
     echo "$i: ${names[$i]"
     # do some processing with ${names[$i]}
done

I get this output:

>_
 0: (Netgear)
 1: (Hon
 2: Hai

but what I want is:

>_
 0: (Netgear)
 1: (Hon Hai Precision Ind. Co.)
 2: (Apple)

I could not figure out a good way to do this, please note that the second string has spaces in it.

Answer 1

Set IFS (Internal Field Separator). Shell uses the IFS variable to determine what the field separators are. By default, IFS is set to the space character. Change it to the newline character, as demonstrated below:

#!/bin/bash
names="Netgear
Hon Hai Precision Ind. Co.
Apple"
    
SAVEIFS=$IFS   # Save current IFS (Internal Field Separator)
IFS=$'\n'      # Change IFS to newline char
names=($names) # split the `names` string into an array by the same name
IFS=$SAVEIFS   # Restore original IFS

for (( i=0; i<${#names[@]}; i++ ))
do
    echo "$i: ${names[$i]}"
done

Output

0: Netgear
1: Hon Hai Precision Ind. Co.
2: Apple

Answer 2

Bash also has a readarray builtin command, easily searchable in the man page. It uses newline (\n) as the default delimiter, and MAPFILE as the default array, so one can do just like so:

    names="Netgear
    Hon Hai Precision Ind. Co.
    Apple"

    readarray -t <<<$names

    printf "0: ${MAPFILE[0]}\n1: ${MAPFILE[1]}\n2: ${MAPFILE[2]}\n"

The -t option removes the delimiter ('\n'), so that it can be explicitly added in printf. The output is:

    0: Netgear
    1: Hon Hai Precision Ind. Co.
    2: Apple

Answer 3

Let me contribute to Sanket Parmar's answer. If you can extract string splitting and processing into a separate function, there is no need to save and restore $IFS — use local instead:

#!/bin/bash

function print_with_line_numbers {
    local IFS=$'\n'
    local lines=($1)
    local i
    for (( i=0; i<${#lines[@]}; i++ )) ; do
        echo "$i: ${lines[$i]}"
    done
}

names="Netgear
Hon Hai Precision Ind. Co.
Apple"

print_with_line_numbers "$names"

See also:

• Setting IFS for a single statement

Answer 4

As others said, IFS will help you.IFS=$'\n' read -ra array <<< "$names" if your variable has string with spaces, put it between double quotes. Now you can easily take all values in a array by ${array[@]}

Answer 5

How to read a multi-line string into a regular bash "indexed" array

The Bash shellcheck static code analyzer and checker tool recommends in SC2206 to use read -r or mapfile. Their mapfile example is complete, but their read example only covers the case of splitting a string by spaces, not newlines, so I learned the complete form of the read command for this purpose from @Toni Dietze's comment here.

So, here is how to use both to split a string by newlines. Note that <<< is called a "herestring". It is similar to << which is a "heredoc", and < which reads in a file:

# split the multiline string stored in variable `var` by newlines, and
# store it into array `myarray`

# Option 1
# - this technique will KEEP empty lines as elements in the array!
# ie: you may end up with some elements being **empty strings**!
mapfile -t myarray <<< "$multiline_string"

# OR: Option 2 [my preference]
# - this technique will NOT keep empty lines as elements in the array!
# ie: you will NOT end up with any elements which are empty strings!
IFS=$'\n' read -r -d '' -a myarray <<< "$multiline_string"

There is also a 3rd technique I use the most, which is not necessarily recommended by shellcheck, but which is fine if you use it correctly, and which is far more readable than either of the options above. I use it in many scripts in my eRCaGuy_dotfiles/useful_scripts directory here. Clone that repo and run grep -rn "IFS" in it to find all places where I use that technique.

See here for where I first learned this: Answer here by @Sanket Parmar: Convert multiline string to array.

Here it is:

# Option 3 [not necessarily recommended by shellcheck perhaps, since you must
# NOT use quotes around the right-hand variable, but it is **much
# easier to read**, and one I very commonly use!]
#
# Convert any multi-line string to an "indexed array" of elements:
#
# See:
# 1. "eRCaGuy_dotfiles/useful_scripts/find_and_replace.sh" for an example 
#    of this.
# 1. *****where I first learned it: https://stackoverflow.com/a/24628676/4561887
SAVEIFS=$IFS   # Save current IFS (Internal Field Separator).
IFS=$'\n'      # Change IFS (Internal Field Separator) to the newline char.
# Split a long string into a bash "indexed array" (via the parenthesis),
# separating by IFS (newline chars); notice that you must intentionally NOT use
# quotes around the parenthesis and variable here for this to work!
myarray=($multiline_string) 
IFS=$SAVEIFS   # Restore IFS

See also:

1. Where I learned my "Option 3" above: Answer here by @Sanket Parmar: Convert multiline string to array
2. Read file into array with empty lines
3. An example where I read a bash multi-line string into a bash array using the read cmd: Find all files in a directory that are not directories themselves

Answer 6

^{Adding the needed null byte delimiter in @HariBharathi answer}

#!/bin/bash

IFS=$'\n' read -r -d '' -a array <<< "$names"

Remark: Unlike mapfile/readarray, this one is compatible with macOS bash 3.2