compare multiple directories on remote hosts

Question

Is there a way to diff on contents of multiple directories instead of two directories? Or diff a single directory on multiple hosts. I wrote the following bash script to diff a directory on three hosts

#!/bin/bash

if [[ -n "$(diff <(ssh user@host1 ls -r /user/test1) <(ssh user@host2 ls -r /user/test1))" || -n "$(diff <(ssh user@host2 ls -r /user/test1) <(ssh user@host3 ls -r /user/test1))" ]]; then
  echo "There are differences"
fi

Is there a better way to do this?

diff itself only works on two at a time. https://stackoverflow.com/questions/572237/whats-the-best-three-way-merge-tool — Calvin Taylor, Sep 05 '17 at 20:02
thanks. I am just wondering if there's a better way to compare multiple directories using a command. — pdna, Sep 05 '17 at 20:07
depends on what you want to find. If `diff` is working, then I wouldn't search further.Good luck. — shellter, Sep 05 '17 at 20:31
Your example is with 2 hosts (and 3 differnt path). Is this a simplified question and do yoy want a solution for 100 host/path combinations? And do you like the output of `rsync -acnv host1:/user/test1 host2 /user/test1` ? — Walter A, Sep 05 '17 at 20:52
@WalterA I would want it for 3-4 hosts. diff does the job for now. yeah I can grab the diff /output but that's not a concern. — pdna, Sep 05 '17 at 21:18

score 3 · Accepted Answer · answered Sep 05 '17 at 23:36

Yes, GNU diff has an option --from-file that allows the comparison of one reference file or directory to many others.

diff -r --from-file=ref-dir dir1 dir2 ... dirN

Note that it will only compare ref-dir to dir1, ..., dirN; it won't compare dir1 to dir2, ..., dirN.

As for your remote directories, since you have ssh access to the machines, you can mount them locally with sshfs in order to execute diff over them.

Maxim Norin · Answer 2 · 2017-09-08T05:15:15.543

You could use MD5 checksums for files lists for each host. It will allow you to use the same script for different count of servers. If lists are the same, you should receive the same values for checksums. And then you just compare all the sums with the previous one. If it differs from any other checksum, then you have differences.

#!/bin/bash

MD5SUMS=$(
for hostname in host{1,2,3}
do
    result=$(ssh user@${hostname} ls -r /user/test1 | md5sum)
    result=${result%% *}
done
)

PREVSUM=""
for SUM in ${MD5SUMS}
do
    if [ -z "$PREVSUM" ]
    then
        PREVSUM=$SUM
        continue
    else
        if [ "$PREVSUM" != "$SUM" ]
        then
            echo "There are differences"
        fi
        PREVSUM=$SUM
    fi
done

compare multiple directories on remote hosts

2 Answers2