I have a while loop that should iterate over a text file but stops at the first line and I can't figure out why. My code is below.
while read hadoop_accounts; do
if ! grep "no lock no remove"; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >> missing_no_lock_no_remove.txt
fi
done < hadoop_accounts.txt
When grep is run with no explicit redirection or file to read, it reads stdin. All of stdin. The same stdin your while read loop is reading from.
Thus, with all your stdin consumed by grep, there's nothing left for the next read command to consume.
The easy approach (and much better for performance) is to do the substring check internal to the shell, and not bother starting up a new copy of grep per line processed at all:
while IFS= read -r hadoop_accounts; do
if ! [[ $hadoop_accounts = *"no lock no remove"* ]]; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
fi
done < hadoop_accounts.txt 3>> missing_no_lock_no_remove.txt
Note also that we're only opening the output file once, not re-opening it every single time we want to write a single line.
If you really want to call grep over and over and over with only a single line of input each time, though, you can do that:
while IFS= read -r hadoop_accounts; do
if ! grep "no lock no remove" <<<"$hadoop_accounts"; then
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
fi
done < hadoop_accounts.txt 3>> missing_no_lock_no_remove.txt
Or, even better than either of the above, you can just run grep a single time over the entire input file and read its output in the loop:
while IFS= read -r hadoop_accounts; do
echo "${hadoop_accounts%%:*}"
echo "${hadoop_accounts%%:*}" >&3
done < <(grep -v 'no lock no remove' <hadoop_accounts.txt) 3>>missing_flags.txt
