How to split quoted argument in shell script

Question

I am writing a shell script. The number of its argument is one, and the only argument is a doubly-quoted string containing spaces like this:

$ ./test.sh "'a bcd   e' 'f ghi' 'jkl mn'"

(These are required specifications and cannot be changed.)

I want to get the following output for the above input.

a bcd   e
f ghi
jkl mn

However, I cannot get this result by using a simple for loop.

In case of Bash

shell script source

#!/bin/bash
for STR in $1; do
    echo $STR
done

result

$ ./test.sh "'a bcd   e' 'f ghi' 'jkl mn'"
'a
bcd
e'
'f
ghi'
'jkl
mn'

In case of Zsh

shell script source

#!/bin/zsh
for STR in $1; do
    echo $STR
done

result

$ ./test.sh "'a bcd   e' 'f ghi' 'jkl mn'"
'a bcd   e' 'f ghi' 'jkl mn'

How can I get the expected output?

Answer 1

Usiing xargs to split quoted arguments and run script itself again with split arguments:

#!/usr/bin/env sh

# If there is only one argument
if  [ "$#" -eq 1 ]; then
  # Use xargs to split arguments
  # and run itself with prepended dummy _ argument
  printf '%s' "$1" | xargs "$0" _
fi

# Remove/ignore prepend dummy argument
shift

i=0
for arg; do
  i=$((i + 1))
  printf 'Arg %d: %s\n' "$i" "$arg"
done

Output from the script with sample data:

$ sh a.sh "'a bcd   e' 'f ghi' 'jkl mn'"
Arg 1: a bcd   e
Arg 2: f ghi
Arg 3: jkl mn

A very shortened version of the argument split and call self:

# Split quoted arguments and call self
[ "$2" ]||printf '%s' "$1"|xargs "$0" _;shift
# Process normal arguments as with any shell script

Answer 2

Another way to handle this is with awk used with ' as the field-separator and then empty fields are discarded outputting the non-empty fields (the separated arguments).

For example, you can do:

#!/bin/bash

awk -F "'" '{
  for (i=1; i<=NF; i++) {
    match ($i, /[^[:blank:]]/)
    if (RSTART >0)
      print $i
  }
}' <<< $1

The awk script loops over each field separated by ' and uses match() to check if a non-blank character is present. (match() sets RSTART to the 1's based index of the first non-blank, so testing if RSTART > 0 tells you a non-blank is present)

Example Use/Output

$ bash test.sh "'a bcd   e' 'f ghi' 'jkl mn'"
a bcd   e
f ghi
jkl mn

If you like, you can use mapfile -t wrapping the awk command above in a process substitution to store the separate arguments in an array if you need to preserve them for later recall.

Since awk does not depend on which shell you have, this is portable to all shells (you would have to pipe the argument for those shells without herestring support.)

Embedded Quotes Lose Meaning To The Shell

I know you said you are stuck with providing the argument to your script in this manner. Know that is the wrong way to go. It is a BashFAQ #50 issue. The single-quotes within double-quotes lose any meaning as a special delimiting character and are simply treated as an embedded ASCII single-quote. This complicates using the command-line.

Forcing that command-line to work isn't a solution, it's Work-Around. The solution is to fix the way the arguments are being provided to your script.

Answer 3

You can use bash regular expression :

#!/usr/bin/env bash

string=$1
pattern="[^']*'([^']+)'"

while [[ $string =~ $pattern ]]; do
    echo "Current argument : ${BASH_REMATCH[1]}"
    string=${string:${#BASH_REMATCH[0]}}
done

Run with :

./test.sh "'a bcd   e' 'f ghi' 'jkl mn'"

Answer 4

Version for zsh:

#!/usr/bin/env zsh
for str in ${(M)${(ps"'")argv}:#*[! ]*}; do
    print $str
done

This uses zsh expansions to split the input into an array on ', and then remove any members of the array that contain only spaces.

./test.zsh "'a bcd   e' 'f ghi' 'jkl mn'"  " '  leading space'"
a bcd   e
f ghi
jkl mn
  leading space