how to format a variable from text file in bash?

Question

I have been stuck on this for while , and tried many ways but couldn't make it work.

let's say I have a text file , with content as follow

mark_line
variable1
value1
value2
mark_line
variable2
value3
mark_line
variable3
value4
value5
value6
value7
mark_line
...
...
mark_line

and so on , basically between each mark_line , the first line is variable name , and rest are values to that variable , which could be one line or multiple lines

expecting output to become like

variable1="value1 value2"

variable2="value3"

varaible3="value4 value5 value6 value7"

and so on ...

I figure this will need a while or for loop , but I couldn't work out proper way for it.

Any hint or thought would be very much appreciated.

"tried many ways but couldn't make it work". Please show your best attempt and explain what issues you have with it. Without that we can't know exactly which aspect of the problem you are having difficulty with. — kaylum, Jan 22 '20 at 23:34
You may want to have a look at: [eval command in Bash and its typical uses](https://stackoverflow.com/questions/11065077/eval-command-in-bash-and-its-typical-uses) — kaylum, Jan 22 '20 at 23:37
@qtwrk : You have tagged your question by _shell_, _posix_ and _bash_. Please make up your mind, whether you want a bash-solution, or need a posix-shell-compatible one. — user1934428, Jan 23 '20 at 10:41
@qtwrk : Also it is not clear from your question, at which point you are stuck: Is it reading a text file, or is it about dynamically generating a shell variable? — user1934428, Jan 23 '20 at 10:43

markp-fuso · Accepted Answer · 2020-01-23T00:15:14.523

Going solely by the requirement to generate a bunch of assignment statements (but not necessarily execute said statements), we'll start with our sample data file:

$ cat mark_line.dat
mark_line
variable1
value1
value2
mark_line
variable2
value3
mark_line
variable3
value4
value5
value6
value7
mark_line
variable4
value8
mark_line
mark_line
mark_line

Next is a bit of awk code that prints variable/value entries to a single line, generating a new line character (\n) only when we see a mark_line entry:

$ awk '
/mark_line/ { printf "\n" ; next }
            { printf $0" "}
' mark_line.dat

<blank_line>
variable1 value1 value2
variable2 value3
variable3 value4 value5 value6 value7
variable4 value8
<blank_line>
<blank_line>

We'll then parse out the blank lines and have a while loop add the equal sign and double quotes to our output:

$ awk '/mark_line/ { printf "\n" ; next } { printf $0" "}' mark_line.dat | \
egrep -v '^$' | \
while read -r myvar myvals
do
    echo "${myvar}=\"${myvals}\""
done

variable1="value1 value2"
variable2="value3"
variable3="value4 value5 value6 value7"
variable4="value8"

NOTE: This solution assumes the each variable? has at least one value? associated with it otherwise a variable? without at least one value? will generate the output: variable5=""

From here I assume the OP already has a plan on how to use this output (eg, perhaps use the eval command per @kaylum's comment?).

very cool, I was trying to read the line number contains "mark_line" , create a for loop between those 2 numbers , and get first "line_number +1" string as variable name , and rest of them as values , but seems **for i in {1...3}** context doesn't allow me to use ${number} variable in it so I was stuck there. also combined with sed to remove **variable=""** empty string :) — qtwrk, Jan 23 '20 at 00:37

score 0 · Answer 2 · answered Jan 23 '20 at 00:59

Would you try the following:

awk -v RS="mark_line\n" -v FS="\n"  '
        /[^[:blank:]]/ {                # skip blank lines
                sub("\n$", "", $0)      # remove trailing newline
                printf("%s=", $1)       # print "variablen=" portion
                str = $2
                for (i=3; i<=NF; i++) str = str " " $i
                print "\"" str "\""     # print "value1 value2 .." portion
        }
' file

score 0 · Answer 3 · answered Jan 23 '20 at 11:57

Printf can write to var, lets use it

$ printf --help
printf: printf [-v var] format [arguments]
    Formats and prints ARGUMENTS under control of the FORMAT.

    Options:
      -v var    assign the output to shell variable VAR rather than
            display it on the standard output
...

Created test file

$ cat test
mark_line
variable1
value1
value2
mark_line
variable2
value3
mark_line
variable3
value4
value5
value6
value7

And the script to process it

#!/bin/bash

data=$(cat test)      # load test file to this var
delimiter='mark_line' # set  delimiter to 'mark_line'

# we'll loop through data with this function
var_creator () {
    while [[ $@ ]]; do
          # if 1st_arg='mark_line', remember 2nd_arg as new_var, add to vars array, remove 2 args
          [[ $1 == "$delimiter" ]] && { new_var=$2; vars+=($2); shift 2; }
          printf -v $new_var "${!new_var} $1" # the 1st_arg is our value coz we shift args, add it to new_var
          shift # remove 1st arg
    done
    # and this one need to remove spaces in front of vars and to print them
    for var in ${vars[@]}; { printf -v $var "${!var# }"; printf "$var=\"${!var# }\"\n"; }
}

var_creator $data # run process

The output

variable1="value1 value2"
variable2="value3"
variable3="value4 value5 value6 value7"

how to format a variable from text file in bash?

3 Answers3