Using array inside awk in shell script

Question

I am very new to Unix shell script and trying to get some knowledge in shell scripting. Please check my requirement and my approach.

I have a input file having data

ABC = A:3 E:3 PS:6 
PQR = B:5 S:5 AS:2 N:2

I am trying to parse the data and get the result as

ABC
A=3
E=3
PS=6

PQR
B=5
S=5
AS=2
N=2

The values can be added horizontally and vertically so I am trying to use an array. I am trying something like this:

myarr=(main.conf | awk -F"=" 'NR!=1 {print $1}'))
echo ${myarr[1]}
# Or loop through every element in the array
for i in "${myarr[@]}"
do
   :
  echo $i
done

or

awk -F"=" 'NR!=1 {
    print $1"\n"
    STR=$2
    IFS=':' read -r -a array <<< "$STR"
    for i in "${!array[@]}"
    do
    echo "$i=>${array[i]}"
    done

}' main.conf

But when I add this code to a .sh file and try to run it, I get syntax errors as

$ awk -F"=" 'NR!=1 {
> print $1"\n"
> STR=$2
> FS=  read -r -a array <<< "$STR"
> for i in "${!array[@]}"
> do
>     echo "$i=>${array[i]}"
> done
>
> }' main.conf
awk: cmd. line:4: FS=  read -r -a array <<< "$STR"
awk: cmd. line:4:                        ^ syntax error
awk: cmd. line:5: for i in "${!array[@]}"
awk: cmd. line:5:     ^ syntax error
awk: cmd. line:8: done
awk: cmd. line:8: ^ syntax error

How can I complete the above expectations?

You're conflating `awk` and `bash/ksh` syntax into one script. `awk` won't understand `IFS` or `<<<` or `read` for the same as shell syntax. Or `${!array[@]}` for that matter. There shouldn't be any problem writing all of this (for your expected output) in just plain `awk`. Throw out the shell syntax and simplify! Look for examples of `awk`'s `splilt()` function to parse your `$2` fields of `A:3 E:3 PS:6`. Good luck. — shellter, Mar 05 '16 at 04:40

score 5 · Answer 1 · edited Apr 13 '17 at 12:36

This is the awk code to do what you want:

$ cat tst.awk
BEGIN { FS="[ =:]+"; OFS="=" }
{
    print $1
    for (i=2;i<NF;i+=2) {
        print $i, $(i+1)
    }
    print ""
}

and this is the shell script (yes, all a shell script does to manipulate text is call awk):

$ awk -f tst.awk file
ABC
A=3
E=3
PS=6

PQR
B=5
S=5
AS=2
N=2

A UNIX shell is an environment from which to call UNIX tools (find, sort, sed, grep, awk, tr, cut, etc.). It has its own language for manipulating (e.g. creating/destroying) files and processes and sequencing calls to tools but it is NOT intended to be used to manipulate text. The guys who invented shell also invented awk for shell to call to manipulate text.

Read https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice and the book Effective Awk Programming, 4th Edition, by Arnold Robbins.

score 2 · Answer 2 · edited May 23 '17 at 11:46

First off, a command that does what you want:

$ sed 's/ = /\n/;y/: /=\n/' main.conf
ABC
A=3
E=3
PS=6

PQR
B=5
S=5
AS=2
N=2

This replaces, on each line, the first (and only) occurrence of = with a newline (the s command), then turns all : into = and all spaces into newlines (the y command). Notice that

this works only because there is a space at the end of the first line (otherwise it would be a bit more involved to get the empty line between the blocks) and
this works only with GNU sed because it substitutes newlines; see this fantastic answer for all the details and how to get it to work with BSD sed.

As for what you tried, there is almost too much wrong with it to try and fix it piece by piece: from the wild mixing of awk and Bash to syntax errors all over the place. I recommend you read good tutorials for both, for example:

The BashGuide
Effective AWK Programming

A Bash solution

Here is a way to solve the same in Bash; I didn't use any arrays.

#!/bin/bash

# Read line by line into the 'line' variable. Setting 'IFS' to the empty string
# preserves leading and trailing whitespace; '-r' prevents interpretation of
# backslash escapes
while IFS= read -r line; do

    # Three parameter expansions:
    # Replace ' = ' by newline (escape backslash)
    line="${line/ = /\\n}"

    # Replace ':' by '='
    line="${line//:/=}"

    # Replace spaces by newlines (escape backslash)
    line="${line// /\\n}"

    # Print the modified input line; '%b' expands backslash escapes
    printf "%b" "$line"
done < "$1"

Output:

$ ./SO.sh main.conf
ABC
A=3
E=3
PS=6

PQR
B=5
S=5
AS=2
N=2

Hey, Thanks for the 2 solutions . It worked for me :) – user2183017 Mar 07 '16 at 12:44 — user2183017, Mar 07 '16 at 12:44

Using array inside awk in shell script

2 Answers2

A Bash solution