3

We are having problems with a client's SAN storage and files "vanishing" whenever the storage synchronizes. We have a custom 4D database that is executing a simple script to sync files from one location to another, via rsync.

The script we are executing is this: "rsync -rvuE --log-file=/tmp/rsync.log SRC DST". The problem is that rsync reports "rsync warning: some files vanished before they could be transferred (code 23)". This error only shows up in terminal/STDOUT and system.log. It doesn't however show up in the --log-file location. I'd like to send it to rsync.log because we read back the log for completion and errors and report it back to the user.

Now here is the tricky part, we are unable to redirect STDOUT or STDERR to the log because it locks up the server.

Noctis Skytower
  • 21,433
  • 16
  • 79
  • 117
Nathan
  • 31
  • 1
  • 3
  • `rsync` on a live data set that is being modified is nearly guaranteed to run into issues like this at some point. Shut down all programs that might modify the data store before you sync it, or use some sort of snapshot technology (will still probably require a brief shutdown/quiesce, anyway). – twalberg Apr 23 '13 at 20:18

3 Answers3

0

Did you try nohup? Which will capture stdout and stderr? You could also grep in the system.log file for the error message and then append that to the log file.

kkron
  • 564
  • 3
  • 20
0

I perform backups with rsync multiple times a day (through an automated tool). I did a grep through my backup logs. My logs show that rsync logs vanished files into its log file by default. I do not use --log-file-format or any other option that changes the format of the log file.

It looks like this:

2012/08/17 19:00:28 [12861] file has vanished: "foo"

I can find other errors logged in there too, like files that could not be transferred due to permissions. The date above is the actual date in the oldest log file I still have that shows that type of error. The version of rsync I used at that time was 3.0.9, so rsync has been doing it since that time.

Louis
  • 146,715
  • 28
  • 274
  • 320
-1

4D is generating .tmp files if there is a memory shortage - and deletes them again. This happens in the 4D data folder. Exclude it from your sync. Just sync the backup files 4D generates. Sync'ing an open 4D datafile fails anyhow... the server is writing to it all the time.

Peter

PeterInWiesbaden
  • 167
  • 3