Passing arguments with spaces contained in variable in bash

Question

Consider passing some set of arguments to a function, e.g.:

awk -vv1="Hello there" -vv2=Bye 'BEGIN {print v1,v2}'

Note that the first argument contains a space. The output is:

Hello there Bye

Now, I try to store the arguments in a variable. Consider first the case without a space in the first argument:

arg="-vv1=Hello -vv2=Bye"
awk $arg 'BEGIN {print v1,v2}'

This works fine. Now insert the space in the variable:

arg="-vv1='Hello there' -vv2=Bye"
awk $arg 'BEGIN {print v1,v2}'

which gives the error message:

awk: there'
awk:      ^ invalid char ''' in expression

What is happening here?

Update

Based on the comments so far, I would like to refine my question: Consider the case where arg is the output of another shell script. For instance, assume that genArg is a shell script that prints the string:

"-vv1='Hello there' -vv2=Bye"

to the terminal. Then arg is collected (in a script) like

arg=$(genArg)

and next I would like to call awk with the given arguments:

awk $arg 'BEGIN {print v1,v2}'

wrt your "Update" - don't do that as that tightly couples your genArg script to your awk script. For example, if 2 years from now you decide to replace your awk script with a perl script or something, you'd need to change genArg - that is tight coupling and should be avoided. Let genArg just output whatever values you care about (e.g. `Hello there` and `Bye` on separate lines) and let your calling shell script then invoke awk (or perl or whatever) as appropriate with those values. Doing that keeps your scripts nicely decoupled and co-incidentally solves your current problem. — Ed Morton, Oct 11 '13 at 21:17
Thanks. How can that be done in practice? How do you invoke `awk` with those values? — Håkon Hægland, Oct 11 '13 at 21:29

score 9 · Answer 1 · answered Oct 11 '13 at 12:37

9

Use an array:

arg=(-vv1="Hello there" -vv2=Bye)
awk "${arg[@]}" 'BEGIN {print v1,v2}'

answered Oct 11 '13 at 12:37

chepner

497,756
71
530
681

1

This is safer, I agree. – Oct 11 '13 at 13:18
In my case the arguments are captured as output from a another command. E.g. `arg=$(echo "-vv1=Hello -vv2=Bye)`. Is it possible to store such output in an array without using `eval`? – Håkon Hægland Oct 11 '13 at 16:11
Unfortunately, no. If you can, modify the other command to output (like alternating variable-name/variable-value lines, which should work as long as the values can't contain newlines) and build the array dynamically. – chepner Oct 11 '13 at 17:04
@HåkonHægland - update your question to show an example of what you're talking about in your comment above. – Ed Morton Oct 11 '13 at 18:15

Ed Morton · Accepted Answer · 2013-10-13T15:44:12.140

3

In response to your updated question and comments:

$ cat tst.sh
function genArg() {
    printf "Hello there\n"
    printf "Bye\n"
}

IFS=$'\n' rslts=( $(genArg) )

awk -v v1="${rslts[0]}" -v v2="${rslts[1]}" 'BEGIN{
    printf "v1=\"%s\"\n", v1
    printf "v2=\"%s\"\n", v2
}'
$
$ ./tst.sh
v1="Hello there"
v2="Bye"

Here's a, IMHO, better alternative:

$ cat tst.sh
function genArg() {
    printf "Hello there\n"
    printf "Bye\n"
}

awk -v rslts="$(genArg)" 'BEGIN{
    split(rslts,v,/\n/)
    for (i=1;i in v;i++)
        printf "v[%d]=\"%s\"\n", i, v[i]
}'
$
$ ./tst.sh
v[1]="Hello there"
v[2]="Bye"

If you want to use named instead of numbered variables:

$ cat ./tst.sh
function genArg() {
    printf "greeting=Hello there\n"
    printf "farewell=Bye\n"
}

awk -v rslts="$(genArg)" 'BEGIN{
    split(rslts,tmp,/\n/)
    for (i=1;i in tmp;i++) {
        var = val = tmp[i]
        sub(/=[^=]+/,"",var)
        sub(/[^=]+=/,"",val)
        v[var] = val
    }

    for (var in v)
        printf "v[%s]=\"%s\"\n", var, v[var]

    greeting = v["greeting"]
    farewell = v["farewell"]

    print "greeting=\"" greeting "\""
    print "farewell=\"" farewell "\""
}'
$
$ ./tst.sh
v[greeting]="Hello there"
v[farewell]="Bye"
greeting="Hello there"
farewell="Bye"

Us the array v[] indexed by key strings as-is or map it's values to variables specifically named based on the keys strings. Either way...

edited Oct 13 '13 at 15:44

answered Oct 12 '13 at 01:07

Ed Morton

188,023
17
78
185

Thanks! Seems like that the `IFS` is essential here. You can even put the options into `genArg` using `printf "%s\n" "-vv1=Hello there"`, and then use `awk "${rslts[@]}"` when calling `awk`. (You could also replace `printf` with `echo`) – Håkon Hægland Oct 12 '13 at 07:06
Yes, setting IFS is necessary or the output of genArg will be split on any white space rather than just at newlines. Don't have genArg include the awk options in its output to avoid tight coupling as I explained previously. Don't replace printf with echo as echo is non-portable and error-prone. – Ed Morton Oct 12 '13 at 13:00
I see your point on tight coupling, however I think that in some cases, like the one I have in mind, if the number of arguments to the `awk` program gets large it is helpful to have a script generate them, avoiding having to type them manually. Then if I later change the awk program to a perl program, I see that I would have to change the `genArg` script.. but that is a risk I am willing to take :) It should not be a too big difficulty to change that script so that it formats its output differently.. – Håkon Hægland Oct 12 '13 at 13:49
1

I'm not suggesting you don't generate the data, just that you don't additionally format it specifically as awk arguments in the same function that generates the data. I updated my answer to show an alternative approach that IMHO is a better way of getting the functionality you seem to want. – Ed Morton Oct 13 '13 at 13:10
Thanks. This is indeed an interesting solution.. But I now worry that we just removed one issue to add another. Now the position of the arguments in the argument list seems to be important in order for `awk` to recognize them correctly. (When the names were included explicitly, this was no problem).. – Håkon Hægland Oct 13 '13 at 13:37
1

The names you used were v1, v2, etc. so I had no idea that wasn't what you were really using. I've posted another iteration of answer. – Ed Morton Oct 13 '13 at 15:44

score 0 · Answer 3 · answered Oct 11 '13 at 11:42

0

Try this:

arg1="-vv1=Hello there"
arg2="-vv2=Bye"
awk "$arg1" "$arg2" 'BEGIN {print v1,v2}'

The idea is to make sure that awk sees exactly three string arguments: vv1, vv2, and the actual awk script text. You can do it this way with two variables, or if you really want to, you could probably also do it using a Bash array.

answered Oct 11 '13 at 11:42

John Zwinck

239,568
38
324
436

Thanks. This works. But my problem is quite general. So I need to consider `N` arguments, and I cannot split them up manually.. – Håkon Hægland Oct 11 '13 at 11:49

Passing arguments with spaces contained in variable in bash

Update

3 Answers3

Linked