Bash while loop behaves differently when run remotely vs locally

Question

There is a simple log file /tmp/test.log

2019-12-09 15:28:12 UTC [6308] App1
2019-12-09 15:29:08 UTC [1246] App2
2019-12-09 15:30:57 UTC [5541] App3

There is a simple bash script /tmp/test.sh

#!/bin/bash
# 
while read line
do
    echo $line
done < /tmp/test.log

If I run this script locally it works as expected:

$ /tmp/test.sh
2019-12-09 15:28:12 UTC [6308] App1
2019-12-09 15:29:08 UTC [1246] App2
2019-12-09 15:30:57 UTC [5541] App3

but if I run this script via ssh I'm getting something unexpected:

$ ssh 127.0.0.1 "/tmp/test.sh"
2019-12-09 15:28:12 UTC 0 3 App1
2019-12-09 15:29:08 UTC 1 App2
2019-12-09 15:30:57 UTC 1 App3

Does anybody know why?

looks like it is interpolating the array in the read line which is odd. — shaine, Dec 10 '19 at 08:44
Can't reproduce on my machine, maybe some execution has run? Try `bash -x` or some equivalent? — Geno Chen, Dec 10 '19 at 08:49
Do you use the same versions of Bash on local & remote machines? — jakub-olczyk, Dec 10 '19 at 09:02
As you can see I connect to 127.0.0.1, so the bash version is the same. — Yuriy Koryk, Dec 10 '19 at 09:40
Aside from special treatment by `echo`, your script essentially does a `cat /tmp/test.log`. So, if instead of doing the loop, you simply do the `cat` in your script, do you still see the differences in the output? How does /tmp/test.log look like when inspected with `xxd`? — user1934428, Dec 10 '19 at 09:44
Unquoted variables are subject to word splitting and **pathname expansion**. `touch 7; var="[12734]"; echo $var` will also output just `7`. — PesaThe, Dec 10 '19 at 09:48

Darren Smith · Answer 1 · 2019-12-10T11:27:48.167

I think there are a couple of separate things interacting here to cause the confusing result. Here's my best guess explanation.

The value you have for the important IFS variable is probably 20 09 0a (these are the hex codes , for space, tab, newline). This has important consequences on the echo (and also read statements). (use echo -n "$IFS" | hexdump to see values).

The read command will read each line from the file, and add them all into the line variable. It will actually tokenise each line, and add each word to line.

The echo statement will cause evaluation of the value in line. The shell will use IFS to separate the value into separate tokens, and then pass them all to echo, which will print them, separated by spaces. I.e., there is a shell pre-processing step that is hard to observe.

The problem arises for tokens like [6308] etc. The square brackets are used for shell globbing, so when bash encounters something like [6308] it will try to match it to files in the current directory named like 6 or 3 or 0 or 8. So if you have two files named 0 and 3, it will match them both and return the string 0 3.

The reason you have different results when run locally as opposed to via ssh is likely just due to the current working directory in which the script is invoked. When run via ssh, it won't run from the your local current directory. To find the cwd used for the ssh invocation, just put a pwd near the top of your file. I'm guessing that the directory used via the ssh invocation has files named like 0 etc.

The simple fix is to prevent the shell processing the contents of $line, by enclosing it with quotes, as in echo "$line" . You could also adjust IFS, but quoting the string is usual practice.

Bash while loop behaves differently when run remotely vs locally

1 Answers1