0

I am trying to create a shell script that reads a file with parameters for a python script the shell script is supposed to execute in a loop:

#!/bin/bash
while read -r i ; do python catfisher.py $i ; done < fishes

where 'fishes' might contain:

vesper "purplish green" fender
vespa "dimmer grey" "stradivarius veil"

The problem is that the python script's argparser interprets the parameters like so:

python purplish green fender

even when echoing $1 in the bash script outputs: vesper "purplish green" fender

The python script is fine if run manually: python catfisher.py "purplish green" fender so I'm assuming it's my lacking bash script skills that are the culprit, rather than my lacking argparser skills but pray advice if I'm mistaken.

David I
  • 31
  • 1
  • 5

1 Answers1

0

Well, the problem is that to do this 100% correctly, you'd have to define a data format for your fishes file and create a parser for it.

You could approximate your desired results by saying "ok, I'd like anything in fishes to be interpreted like a shell would interpret commands to an argument", but that also entails other metacharacters than spaces and quotes which you already have, such as dollar signs, braces, tildes, parentheses, etc.

Since you are using variable expansion to feed your line to Python, your shell is already past the quoting phase, thus not interpreting quotes contained in the variable.

Unfortunately, there are no mechanisms to selectively carry out shell parsing phases, like you want to do, ignoring dangerous parts. You can pass the text to a new shell, but that allows your data to contain commands, making your software vulnerable to code injection:

while read -r i ; do bash -c "python catfisher.py $i" ; done < fishes

The best solution would probably be close to the first suggestion, except rather than defining your own data format, you could use an existing one, such as CSV, some dialects of which already allow for quoting, and are widely supported by libraries, like the csv module, built into Python's standard library.

Larry
  • 427
  • 2
  • 10
  • Thanks for the suggestion and explanation of how bash works here; I think I'll skip the bash shell script and add file reading to the python script instead. – David I Sep 23 '21 at 15:32
  • @DavidI I honestly don't have enough information about what role your command line argument file plays in the grand scheme. If it is not a data file like I assumed, you could also just turn it into a command file directly and prepend "python " to every line, to be executed by a shell. – Larry Sep 23 '21 at 20:51