0

I'm writing a shell script intended to edit audio files using the sox command. I've been running into a strange problem I never encountered in bash scripting before: When defining space separated effects in sox, the command will work when that effect is written directly, but not when it's stored in a variable. This means the following works fine and without any issues:

sox ./test.in.wav ./test.out.wav delay 5

Yet for some reason the following will not work:

IFS='   ' # set IFS to only have a tab character because file is tab-separated
while read -r file effects text; do
  sox $file.in.wav $file.out.wav $effects
done <in.txt

...when its in.txt is created with:

printf '%s\t%s\t%s\n' "test" "delay 5" "other text here" >in.txt

The error indicates this is causing it to see the output file as another input.

sox FAIL formats: can't open input file `./output.wav': No such file or directory

I tried everything I could think of: Using quotation marks (sox "$file.in.wav" "$file.out.wav" "$effects"), echoing the variable in-line (sox $file.in.wav $file.out.wav $(echo $effects)), even escaping the space inside the variable (effects="delay\ 5"). Nothing seems to work, everything produces the error. Why does one command work but not the other, what am I missing and how do I solve it?

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
MirceaKitsune
  • 777
  • 1
  • 5
  • 14
  • I tried this, that's why it's weird. Both `sox ./input.wav ./output.wav "$effects"` and `sox ./input.wav ./output.wav '$effects'` produce the same problem. – MirceaKitsune Sep 03 '21 at 21:35
  • 1
    Right, use an array `effects=(delay 5)` then to use it `"${effects[@]}"` – Jetchisel Sep 03 '21 at 21:37
  • That appears to work! Only one issue remains: The actual variable is read from a text file where it's written using spaces as intended. Is there a quick way to convert the space separated list into an array in the same line? – MirceaKitsune Sep 03 '21 at 21:41
  • fwiw, enable debug to see how `$effects` is being passed to `sox`, eg: `set -xv ; sox ... ; set+xv` – markp-fuso Sep 03 '21 at 21:42
  • What's the current value of `IFS`? – Charles Duffy Sep 03 '21 at 21:42
  • I'd suggest updating the question with the sample 'text file' (where the variable is being populated from) so we can see what we're dealing with – markp-fuso Sep 03 '21 at 21:43
  • `IFS=' ' read -r -a effects – Charles Duffy Sep 03 '21 at 21:44
  • Better would be one line per argument, and to use `readarray -t effects – Charles Duffy Sep 03 '21 at 21:44
  • Aha: `set -xv` / `set +xv` indicates the command is seen as `sox ./input.wav ./output.wav 'delay 5'`, the presence of those '' characters definitely isn't right. Why is bash automatically adding those when parsing the variable? – MirceaKitsune Sep 03 '21 at 21:45
  • 2
    The characters are just the shell showing you that `delay 5` is all one argument. It's not that something you don't want is being added; it's debug output letting you distinguish between all-one-argument and multiple-separate-arguments. – Charles Duffy Sep 03 '21 at 21:47
  • As for the "why", see [BashFAQ #50](https://mywiki.wooledge.org/BashFAQ/050), and read what I said earlier about `IFS`. – Charles Duffy Sep 03 '21 at 21:47
  • BTW, see [Reading quoted/escaped arguments correctly from a string](https://stackoverflow.com/questions/26067249/reading-quoted-escaped-arguments-correctly-from-a-string) – Charles Duffy Sep 03 '21 at 21:49
  • (You haven't answered the question I asked about whether anything else in your script is modifying IFS before the lines in question get run, but there are only two possibilities -- IFS is being changed from its defaults, or the shell executing the script isn't bash -- and also, for that matter, isn't POSIX-compliant. BTW, the most popular recent shell that _isn't_ POSIX-compliant is zsh). – Charles Duffy Sep 03 '21 at 21:50
  • As far as the full script goes it's a bit more complex, I'll update the question and share a simplified version if necessary. In essence I'm using the read command to parse audio settings per line from a tab separated text file, hence why I set `IFS=" "` which IIRC only affects the `read` command to work as I intend, the variable in cause is fetched in `while read -r file effects text; do ... done < ./file.txt` loop. – MirceaKitsune Sep 03 '21 at 21:50
  • 1
    If you want your `IFS` change to only modify `read`, then put it in your `while read` line, as in, `while IFS=$'\t' read -r file effects text; do ...`, and don't change `IFS` anywhere else -- if you do it that way the change is scoped to the `read`. – Charles Duffy Sep 03 '21 at 21:52
  • 1
    (no, `IFS` does not only change `read`; it also changes how _all_ unquoted expansions work, hence modifying the meaning of `$effects`; it doesn't change `"$effects"`, but the behavior the quotes enforce is the behavior you don't want). – Charles Duffy Sep 03 '21 at 21:53

1 Answers1

1

IFS does not only change the behavior of read; it also changes the behavior of unquoted expansions.

In particular, unquoted expansions' content are split on characters found in IFS, before each element resulting from that split is expanded as a glob.

Thus, if you want the space between delay and 5 to be used for word splitting, you need to have a regular space, not just a tab, in IFS. If you move your IFS assignment to be part of the same simple command as the read, as in IFS=$'\t' read -r file effects text; do, that will stop it from changing behavior in the rest of the script.

However, it's not good practice to use unquoted expansions for word-splitting at all. Use an array instead. You can split your effects string into an array with:

IFS=' ' read -r -a effects_arr <<<"$effects"

...and then run sox "$file.in.wav" "$file.out.wav" "${effects_arr[@]}" to expand each item in the array as a separate word.

By contrast, if you need quotes/escapes/etc to be allowed in effects, see Reading quoted/escaped arguments correctly from a string

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • Thank you, that explains it. I read about the `read` command before using it and the articles said I can modify the `IFS` variable if I want to change the delimiter it's using, yet they didn't mention it changes anything else. I'm trying the line suggested in an earlier comment: `while IFS=$'\t' read -r file effects text; do`. This seems to be breaking if statements inside the loop for some reason, looking into that but otherwise it looks like it might have solved it. – MirceaKitsune Sep 03 '21 at 22:02
  • 1
    I'd be curious to see exactly what within the loop is misbehaving -- you might need more quotes somewhere if you're using an unquoted expansion and assuming that your expansion won't be word-split into more than one argument. Consider running your code through http://shellcheck.net/ and seeing what it finds. – Charles Duffy Sep 03 '21 at 22:03
  • @MirceaKitsune, ...for example, if you have `if [ $key = $value ]`, that's more safely written as `if [ "$key" = "$value" ]`, which provides assurance that both `key` and `value` will expand to exactly one word each (because if it's more or less than that, the result may no longer be valid syntax for the `[` command). – Charles Duffy Sep 03 '21 at 22:08
  • Fixed the last issue: `if [ ! -z $text ]` had to be replaced with `if [ -n "$text" ]`. It now works without any problem after setting `IFS` inside the while loop instead of globally before it! Thank you for the solution. – MirceaKitsune Sep 03 '21 at 22:09
  • 1
    Great! `[ ! -z "$text" ]` should work too, but `[ -n "$text" ]` is definitely clearer. – Charles Duffy Sep 03 '21 at 22:11