How to loop over a file and compare one file in them with a backup file and check if changed

Question

This is my run.bash file (in which I have problem that I can't iterate over the file completely if I remove $line = ... ):


#!/usr/bin/bash

while IFS= read -r line
do
    if [ "$(ssh -p2200 "$line" "sha256sum /home/config.conf | awk '{print $1}'")" == "$(ssh -p2200 "$line" "sha256sum /home/config.conf-bk | awk '{print $1}'")" ]
    then
        echo "File is the same in \"$line\"." > /dev/null 2>&1 # this output is not important
    else
        echo "The config.conf file in $line has changed, so we reverted the file." | mail -s "The config.conf file in $line has changed" myemail@gmail.com
        ssh -np2200 "$line" cp -a /home/config.conf-bk /home/config.conf
    fi
done < file.txt

This is file.txt:

G9DG1.fqdn.com
G9DG2.fqdn.com
G9DG3.fqdn.com
G9DG4.fqdn.com
G9DC3.fqdn.com
G9DG5.fqdn.com
G9DC4.fqdn.com

This is the /home/config.conf and /home/config.conf-bk which should be the same on each server:

# this is a test config file

This is sha256sum config.conf:

91c17dcc7bc4c48594c0d8eb9726b97e5e2c6aaf728779c9e39a69fe76229c9e  config.conf

At the moment, the value of sha256sum this file is the same as config.conf-bk file. Now if I run my scripts, it prints:

The config.conf file in G9DG1.fqdn.com has changed, so we reverted the file

I also changed my script to:


#!/usr/bin/bash

while IFS= read -r line
do
    if [[ "$line" =~ ^([Gg][0-9]{1,2}[DS][C][0-9]{1,3}.fqdn.com)$ ]] # C-Plans
    then
        if [ "$(ssh -p2200 "$line" "sha256sum /home/config.conf | awk '{print $1}'")" == "$(ssh -p2200 "$line" "sha256sum /home/config.conf-bk | awk '{print $1}'")" ]
        then
            echo "File is the same in \"$line\"." > /dev/null 2>&1 # this output is not important
        else
            echo "The config.conf file in \"$line\" has changed, so we reverted the file." | mail -s "The config.conf file in \"$line\" has changed" myemail@gmail.com
            ssh -np2200 "$line" cp -a /home/config.conf-bk /home/config.conf
        fi
    elif [[ "$line" =~ ^([Gg][0-9]{1,2}[DS][G][0-9]{1,3}.fqdn.com)$ ]] # G-Plans
    then
        if [ "$(ssh -p2200 \"$line\" "sha256sum /home/config.conf | awk '{print $1}'")" == "$(ssh -p2200 \"$line\" "sha256sum /home/config.conf-bk | awk '{print $1}'")" ]
        then
            echo "File is the same in $line." > /dev/null 2>&1 # this output is not important
        else
            echo "The config.conf file in $line has changed, so we reverted the file." | mail -s "The config.conf file in $line has changed" myemail@gmail.com
            ssh -np2200 "$line" cp -a /home/config.conf-bk /home/config.conf
        fi
    fi
done < file.txt

But this time I see this:

File is the same in G9DG1.fqdn.com.
File is the same in G9DG2.fqdn.com.
File is the same in G9DG3.fqdn.com.
File is the same in G9DG4.fqdn.com.
The config.conf file in G9DC3.fqdn.com has changed, so we reverted the file.

The expected output is to check all servers (not only after the first matching if and terminate the clause) and check if file is changed or not, and then do the if-else clause.

Update 1

The below bash file now loops all over the file, but always echos the else part of both two inner loops:

#!/usr/bin/bash

while IFS= read -r line
do
    if [[ "$line" =~ ^([Gg][0-9]{1,2}[DS][C][0-9]{1,3}.fqdn.com)$ ]] # C-Plans
    then
        if [ "$(ssh -np2200 $line "sha256sum /home/config.conf | awk '{print $1}'")" == "$(ssh -np2200 $line "sha256sum /home/config.conf-bk | awk '{print $1}'")" ]
        then
            echo "It is the same in $line."
        else
            echo "The config.conf file in $line has changed, so we reverted the file."
        fi
    elif [[ "$line" =~ ^([Gg][0-9]{1,2}[DS][G][0-9]{1,3}.fqdn.com)$ ]] # G-Plans
    then
        if [ "$(ssh -np2200 $line "sha256sum /home/config.conf | awk '{print $1}'")" == "$(ssh -np2200 $line "sha256sum /home/config.conf-bk | awk '{print $1}'")" ]
        then
            echo "It is the same in $line."
        else
            echo "The config.conf file in $line has changed, so we reverted the file."
        fi
    fi
done < file.txt

The current (unexpected) output:

It is not the same in G1DG1.fqdn.com.
It is not the same in G1DG2.fqdn.com.
It is not the same in G1DG3.fqdn.com.
It is not the same in G1DG4.fqdn.com.
It is not the same in G1DC3.fqdn.com.
It is not the same in G1DG5.fqdn.com.
It is not the same in G1DC4.fqdn.com.

Please paste your script at [shellcheck.net](http://www.shellcheck.net/). — Cyrus, Jul 02 '23 at 07:59
@Freeman no offense sir, as EdMorton suggested, I removed temporarily the best answer to see if there are more people to answer. But I'll consider that question soon. — Saeed, Jul 02 '23 at 08:03
`sed` without the `-n` option will read from stdin, effectively stealing the rest of the file being read from. This looks like a duplicate of ["While loop stops reading after the first line in Bash"](https://stackoverflow.com/questions/13800225/while-loop-stops-reading-after-the-first-line-in-bash) (also, see [BashFAQ #89: "I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!"](http://mywiki.wooledge.org/BashFAQ/089)). — Gordon Davisson, Jul 02 '23 at 08:24
While a manual comparison is a good exercise, that's what the `diff` utility is for. It can compare files (or all files in a pair of directories) blazingly fast and identify each byte that is changed. — David C. Rankin, Jul 02 '23 at 08:42
@ufopilot D'oh! You're right of course; unfortunately, I can't go back and edit it. — Gordon Davisson, Jul 02 '23 at 09:12
@Saeed Is there an extra ".conf" in your filenames (e.g. "/home/config.conf.conf")? If that's not the problem, put `set -x` before the loop to get an execution trace, and see what the remote commands are coming back with. — Gordon Davisson, Jul 02 '23 at 09:20
@GordonDavisson yes, there's extra `.conf` in the file name in the question, in which the real file doesn't contain two `.conf` name. I edit that. Thanks — Saeed, Jul 02 '23 at 11:01

Kaz · Accepted Answer · 2023-07-02T09:14:27.793

Your escaping is off. You have an awk command that is enclosed in double quotes:

"command | awk '... $1 ...'"

The awk script is in single quotes, but those quotes are dummies; they are not quote syntax, because they are inside a double quote! The $1 is not protected by the single quotes at all; it is exposed to the double quote in which expansions like $1 are perfectly active.

If your script has not been given any arguments, then $1 expands to nothing and your awk code is just { print }, which will include the path name with the SHA256. The path names ar enot equal, and so ...

Try awk '{ print \$1 }' to defeat the special meaning of the dollar sign and preserve $1 so it can get through to the shell on the other side of the ssh connection.

I tried `\$1` before but didn't work. A combination of `set -x` and your answer worked for me, as said in comments by @GordonDavisson — Saeed, Jul 02 '23 at 09:59

Freeman · Answer 2 · 2023-07-02T14:01:54.133

check this out :

while IFS= read -r line
do
    if [[ "$line" =~ ^(G[0-9]{1,2}[DS]C[0-9]{1,3}\.fqdn\.com)$ ]] # C-Plans
    then
        if [ "$(ssh -p2200 "$line" "sha256sum /home/config.conf | awk '{print $1}'")" == "$(ssh -p2200 "$line" "sha256sum /home/config.conf-bk | awk '{print $1}'")" ]
        then
            echo "File is the same in \"$line\"." > /dev/null 2>&1 # this output is not important
        else
            echo "The config.conf file in \"$line\" has changed, so we reverted the file." | mail -s "The config.conf file in \"$line\" has changed" myemail@gmail.com
            ssh -np2200 "$line" cp -a /home/config.conf-bk /home/config.conf
        fi
    elif [[ "$line" =~ ^(G[0-9]{1,2}[DS]G[0-9]{1,3}\.fqdn\.com)$ ]] # G-Plans
    then
        if [ "$(ssh -p2200 "$line" "sha256sum /home/config.conf | awk '{print $1}'")" == "$(ssh -p2200 "$line" "sha256sum /home/config.conf-bk | awk '{print $1}'")" ]
        then
            echo "File is the same in \"$line\"." > /dev/null 2>&1 # this output is not important
        else
            echo "The config.conf file in \"$line\" has changed, so we reverted the file." | mail -s "The config.conf file in \"$line\" has changed" myemail@gmail.com
            ssh -np2200 "$line" cp -a /home/config.conf-bk /home/config.conf
        fi
    fi
done < file.txt

@Saeed an alternative approach involves capturing the output of the `sha256sum` command into the `remote_hash` and `local_hash` variables. this is achieved by utilizing the `read` command within the `ssh` command and the local shell, respectively. by comparing these variables, we can determine whether the files are identical or if any changes have occurred! — Freeman, Jul 02 '23 at 10:44
Thanks, I'll check it soon. I edited the question, there were extra `.conf.conf` which replaced by `.conf`. Please edit the answer and file names — Saeed, Jul 02 '23 at 11:04

How to loop over a file and compare one file in them with a backup file and check if changed

2 Answers2