1

Is there a bash builtin or expression that is equivalent to python's shlex.split? It would be great if it could handle not only whitespace but any IFS.

I have a file with a bunch of simple bash commands. 

cat a_filename
ls a\ filename\ with\ spaces
less "another with spaces"
ln -s "this is" my\ favorite
echo $(I do not mind if this does not work)

What I want to do is to process this file line-by-line by splitting the commands on every line into argument arrays. I can do this in python simple enough with shlex.split:

>>> import shlex
>>> for line in open("/tmp/example"):
...     print(shlex.split(line))
...
['cat', 'a_filename']
['ls', 'a filename with spaces']
['less', 'another with spaces']
['ln', '-s', 'this is', 'my favorite']
['echo', '$(I', 'do', 'not', 'mind', 'if', 'this', 'does', 'not', 'work)']

Please note that I am not interested in non-bash solutions or solutions that use eval or hand-crafted escaping. The following for example is not a good solution, as it allows command injection.

$ cat /tmp/example_line
cat "this is" not\ ok $(date)
$ eval "array=($(cat /tmp/example_line))"
$ echo ${array[@]}
cat this is not ok Sun Jan 13 17:13:44 CET 2019
pgy
  • 291
  • 1
  • 4
  • "Builtin or expression"? No, but you can get *close* relying only on POSIX-mandated behavior in `xargs`. – Charles Duffy Jan 13 '19 at 16:29
  • 1
    BTW, `eval` doesn't have any special parser-level behavior -- you need to quote its arguments same as you quote everything else if you don't want them string-split and glob-expanded (before being concatenated together into a single string which is run through the parser). Not that the above changes your demonstration in any pertinent way, but there are plenty of other examples where `eval` will cause even more misbehavior than usual with unquoted arguments. – Charles Duffy Jan 13 '19 at 16:30
  • Hmm, why did I not think of xargs? Thank you! Yes, you are perfectly right about eval, I just can't resist advocating against using it anytime it's not strictly necessary. – pgy Jan 13 '19 at 16:39

0 Answers0