bash - wget -N if else value check

Question

I'm working on a bash script that pulls a file from an FTP site only if the timestamp on remote is different than local. After it puts the file, it copies the file over to 3 other computers via samba (smbclient).

Everything works, but the file copies even if the wget -N ftp://insertsitehere.com returns a value that the file on the remote was not newer. What would be the best way to check the output of the script so that the copy only happens if a new version was pulled from FTP?

Ideally, I'd like the copy to the computers to preserve the timestamp just like the wget -N command does, too.

Here is an example of what I have:

#!/bin/bash

OUTDIR=/cats/dogs

cd $OUTDIR
wget -N ftp://user:password@sitegoeshere.com/filename

if [ $? -eq 0 ]; then
   HOSTS="server1 server2 server3"
   for i in $HOSTS; do
      echo "Uploading to $i..."
      smbclient -A /root/.smbclient.authfile //$i/path -c "lcd /cats/dogs; put fiilename.txt"
      if [ $? -eq 0 ]; then
        echo "Upload to $i successful..."
      else
        echo "There was an issue uploading to host $i..."
      fi
   done
else
   echo "There was an issue with the FTP Download...."
   exit 1
fi

Select all of your code with your mouse, then click the `{}` (format) tool at the top-left of the edit box to make your code readable. Based on wget man-page, seems like this should work. To debug, I would `ssh user@sitesgoeshere.com '/bin/ls -l filename'` before and confirm that date is really older than current. Are the possible issues with time-zones? Do you need to add an X-hour offset between timestamps from the different systems? Nice first question. Keep posting and good luck. — shellter, Jun 26 '14 at 03:21
The timestamp works from the wget pull, but the second part of the script to copy it over to server1, server2 and serve3 occurs regardless of if the file was downloaded again or not. That's the problem. Also, the copy to each server does not preserve the timestamp that is correct with the wget -N command. — jdeen, Jun 26 '14 at 03:26
Sorry, should have gone to bed hours ago. Do you mean that the test `if [ $? -eq 0 ]` is returning true when it shouldn't? That is the only way the smbclient can be invoked so it must be true. Maybe `-eq 0` is the wrong operator, I use `if wget .... ; then` to eliminate such issues. I don't have access to smbclient. Hopefully perusing `man smbclient` will find you an option like `scp -p` (preserve file permissions (and timestamps). Good luck! — shellter, Jun 26 '14 at 03:30
Thanks! I'll look further in that. And yes, if I echo $? i get a return value of 0 no matter what. Seems like it's saying successful if the file downloads or if it doesn't (because it's obeying the -N switch). I can test updating it to if then, but having some sort of check to say if file wasn't downloaded, then don't copy, if file was downloaded (because file was newer) proceed to copy. (and preserve time stamps) — jdeen, Jun 26 '14 at 03:42
Well the most straight-forward (not as elegant as a proper value in `$?`), might be to `cd /tmp/working; wget ...; if [[ -f newFileVer ]] ; then smbclient ... ; etc ;` and get rid the of file in `/tmp/working` (or however you like to manage working files). Good luck. — shellter, Jun 26 '14 at 11:50

score 3 · Answer 1 · edited Apr 24 '15 at 16:08

The return value of wget is different than 0 only if there is an error. If -N is in use and the remote file is older than the local file, it will still have a return value of 0, so you cannot use that to check if the file has been modified.

You could check the mtime of the file to see if it changed, or the content. For example, you could use something like:

md5_old=$( md5sum filename.txt 2>/dev/null )
wget -N ftp://user:password@sitegoeshere.com/filename.txt
md5_new=$( md5sum filename.txt )

if [ "$md5_old" != "$md5_new" ]; then
    # Copy filename.txt to SMB servers
fi

Regarding smbclient, unfortunately there is no way to preserve timestamps in either get or put commands. If you need it, you must use some different tool (scp -p, rsync -t...)

score 1 · Answer 2 · answered Dec 30 '14 at 05:47

touch -r foo.txt foo.old
wget -N example.com/foo.txt
if [ foo.txt -nt foo.old ]
then
  echo 'Uploading to server1...'
fi

"Save" the current timestamp into a new empty file
Use wget --timestamping to only download the file if it is newer
If file is newer than the "save" file, do stuff

bash - wget -N if else value check

2 Answers2