Bash reading from file and parsing each line

Question

I need to manage updates of a particular software on several customers' linux servers. The updates are provided by sending .tar.gz files to each server and inflating it overwriting the old software files.

I've put the customers' server list in a txt file like this:

user1@customer-server-1-address:22:/path/to/software/server1:NULL:Customer1name

user2@customer-server-2-address:2222:/path/to/software/server2:/etc/vpnc/client1.conf:Customer2name

user3@customer-server-3-address:22:/path/to/software/server3:NULL:Customer3name

Char ":" acts as field separator; field 1 is the server address, 2 is port, 3 is path to software, 4 is used to tell if a VPN tunnel is to be enabled first, 5 is customer name.

I've managed to create the following script:

#!/bin/bash

[...]

while IFS=':' read -ra array; do
    if [ "${array[3]}" != "NULL" ]; then
        echo
        echo "### USING VPN CONF '${array[3]}' FOR SERVER '${array[0]}'... ###"
        sleep 1
        #vpnc ${array[3]}
        TUNNEL1=$(ifconfig | grep -o eth0)
        TUNNEL2='eth0'

        if [ "$TUNNEL1" = "$TUNNEL2" ]; then
          echo "### TUNNEL IS UP ###"

          scp -P${array[1]} $FILENAME.gz ${array[0]}:${array[2]}
          scp -P${array[1]} $FILELIST ${array[0]}:${array[2]}

          ssh -p${array[1]} ${array[0]} 'cd '${array[2]}' && cat '${array[2]}'/'$FILELIST' | cpio --quiet -H ustar -o -0 -L -F '${array[2]}'/'$BACKUP_FILENAME
          ssh -p${array[1]} ${array[0]} 'gzip '${array[2]}'/'$BACKUP_FILENAME

          ssh -p${array[1]} ${array[0]} 'tar -C '${array[2]}' -xf '${array[2]}'/'$FILENAME.gz
          ssh -p${array[1]} ${array[0]} 'cd '${array[2]}' && mkdir -p '$REMOTE_BACKUP_DIRECTORY' && rm -f '$FILELIST' && mv -f '$BACKUP_FILENAME.gz $REMOTE_BACKUP_DIRECTORY

          echo "### SERVER '${array[0]}' UPDATED ###"
          #vpnc-disconnect
          sleep 1
        else
          echo "### VPN ERROR IN SERVER '${array[4]}' ###"
        fi
    else
        echo

        scp -P${array[1]} $FILENAME.gz ${array[0]}:${array[2]}
        scp -P${array[1]} $FILELIST ${array[0]}:${array[2]}

        ssh -p${array[1]} ${array[0]} 'cd '${array[2]}' && cat '${array[2]}'/'$FILELIST' | cpio --quiet -H ustar -o -0 -L -F '${array[2]}'/'$BACKUP_FILENAME-${array[4]}
        ssh -p${array[1]} ${array[0]} 'gzip '${array[2]}'/'$BACKUP_FILENAME-${array[4]}

        ssh -p${array[1]} ${array[0]} 'tar -C '${array[2]}' -xf '${array[2]}'/'$FILENAME.gz

        ssh -p${array[1]} ${array[0]} 'cd '${array[2]}' && mkdir -p '$REMOTE_BACKUP_DIRECTORY' && rm -f '$FILELIST' && mv -f '$BACKUP_FILENAME-${array[4]}.gz $REMOTE_BACKUP_DIRECTORY

        echo "### SERVER '${array[4]}' UPDATED ###"
        echo
    fi;
done < "$CUSTOMER_SERVERS_FILE"

$CUSTOMER_SERVERS_FILE is the container file as described earlier, $FILENAME.gz is the update package, $FILELIST is the list of the files to update. Other variables set for other purposes.

Problem is, this whole thing is effectively running only for the first customer in my $CUSTOMER_SERVERS_FILE. The lines above the first are being skipped, and I can't tell why. Any hints? Thank you.

EDIT: As an answer to the duplicate objection, being the script rather large I didn't think of any link between the while and ssh objects. I was rather thinking more of a syntax error. However, I can still delete the question if asked to do so.

simplify the problem, have it do a single thing so you can figure out why the loop is wrong. i.e. is the problem in the while loop or somewhere in the body of your code... Break the problem into smaller problems and see where you get stuck and why. — UpAndAdam, May 29 '14 at 16:11
PRO TIP: I've just noticed removing all the "ssh" lines solves, but still I can't tell why this is not working — Seether, May 29 '14 at 16:18
So try adding them back one at a time, and see which one you get stuck at... — UpAndAdam, May 29 '14 at 16:20
The first is already enough to break it. But I still really have no idea why... — Seether, May 29 '14 at 16:30
good script. You're doing fine. stdin for while and ssh is an advanced problem. Keep plugging away. Good luck to all. — shellter, May 29 '14 at 16:41

score 4 · Accepted Answer · answered May 29 '14 at 16:27

4

This happens because your while read loop and ssh both read from stdin.

You can fix it by using /dev/null as ssh's stdin:

ssh host cmd  < /dev/null

answered May 29 '14 at 16:27

that other guy

116,971
11
170
194

1

Also `ssh -n host cmd`. – chepner May 29 '14 at 16:27
That was it! Thank you so much, I could have never got that myself. I feel as I'm never going to know enough about bash scripting... – Seether May 29 '14 at 16:35

Bash reading from file and parsing each line

1 Answers1