Archive old files only AND re-construct folder tree in archive

Question

I want to move all my files older than 1000 days, which are distributed over various subfolders, from /home/user/documents into /home/user/archive. The command I tried was

find /home/user/documents -type f -mtime +1000 -exec rsync -a --progress --remove-source-files {} /home/user/archive \;

The problem is, that (understandably) all files end up being moved into the single folder /home/user/archive. However, what I want is to re-construct the file tree below /home/user/documents inside /home/user/archive. I figure this should be possible by simply replacing a string with another somehow, but how? What is the command that serves this purpose?

Thank you!

This may help: http://stackoverflow.com/q/1650164/1983854 – fedorqui Sep 05 '13 at 15:44 — fedorqui, Sep 05 '13 at 15:44

twalberg · Answer 1 · 2013-09-05T18:14:31.570

I would take this route instead of rsync:

Change directories so we can deal with relative path names instead of absolute ones:
```
cd /home/user/documents
```
Run your find command and feed the output to cpio, requesting it to make hard-links (-l) to the files, creating the leading directories (-d) and preserve attributes (-m). The -print0 and -0 options use nulls as record terminators to correctly handle file names with whitespace in them. The -l option to cpio uses links instead of actually copying the files, so very little additional space is used (just what is needed for the new directories).
```
find . -type f -mtime +1000 -print0 | cpio -dumpl0 /home/user/archives
```
Re-run your find command and feed the output to xargs rm to remove the originals:
```
find . -type f -mtime +1000 -print0 | xargs -0 rm
```

score 0 · Answer 2 · answered Sep 05 '13 at 19:37

Here's a script too.

#!/bin/bash

[ -n "$BASH_VERSION" ] && [[ BASH_VERSINFO -ge 4 ]] || {
    echo "You need Bash version 4.0 to run this script."
    exit 1
}

# SOURCE=/home/user/documents/
# DEST=/home/user/archive/

SOURCE=$1
DEST=$2

declare -i DAYSOLD=10
declare -a DIRS=()
declare -A DIRS_HASH=()
declare -a FILES=()
declare -i E=0

# Check directories.

[[ -n $SOURCE && -d $SOURCE && -n $DEST && -d $DEST ]] || {
    echo "Source or destination directory may be invalid."
    exit 1
}

# Format source and dest variables properly:

SOURCE=${SOURCE%/}
DEST=${DEST%/}
SOURCE_LENGTH=${#SOURCE}

# Copy directories first.

echo "Creating directories."

while read -r FILE; do
    DIR=${FILE%/*}
    if [[ -z ${DIRS_HASH[$DIR]} ]]; then
        PARTIAL=${DIR:SOURCE_LENGTH}
        if [[ -n $PARTIAL ]]; then
            TARGET=${DEST}${PARTIAL}
            echo "'$TARGET'"
            mkdir -p "$TARGET" || (( E += $? ))
            chmod --reference="$DIR" "$TARGET" || (( E += $? ))
            chown --reference="$DIR" "$TARGET" || (( E += $? ))
            touch --reference="$DIR" "$TARGET" || (( E += $? ))
            DIRS+=("$DIR")
        fi
        DIRS_HASH[$DIR]=.
    fi
done < <(exec find "$SOURCE" -mindepth 1 -type f -mtime +"$DAYSOLD")

# Copy files.

echo "Copying files."

while read -r FILE; do
    PARTIAL=${FILE:SOURCE_LENGTH}
    cp -av "$FILE" "${DEST}${PARTIAL}" || (( E += $? ))
    FILES+=("$FILE")
done < <(exec find "$SOURCE" -mindepth 1 -type f -mtime +"$DAYSOLD")

# Remove old files.

if [[ E -eq 0 ]]; then
    echo "Removing old files."
    rm -fr "${DIRS[@]}" "${FILES[@]}"
else
    echo "An error occurred during copy. Not removing old files."
    exit 1
fi

Archive old files only AND re-construct folder tree in archive

2 Answers2