Copy folder recursively, excluding some folders

Question

I am trying to write a simple bash script that will copy the entire contents of a folder including hidden files and folders into another folder, but I want to exclude certain specific folders. How could I achieve this?

Answer 1

Use rsync:

rsync -av --exclude='path1/to/exclude' --exclude='path2/to/exclude' source destination

Note that using source and source/ are different. A trailing slash means to copy the contents of the folder source into destination. Without the trailing slash, it means copy the folder source into destination.

Alternatively, if you have lots of directories (or files) to exclude, you can use --exclude-from=FILE, where FILE is the name of a file containing files or directories to exclude.

--exclude may also contain wildcards, such as --exclude=*/.svn*

Answer 2

Use tar along with a pipe.

cd /source_directory
tar cf - --exclude=dir_to_exclude . | (cd /destination && tar xvf - )

You can even use this technique across ssh.

Answer 3

You can use find with the -prune option.

An example from man find:

       cd /source-dir
       find . -name .snapshot -prune -o \( \! -name *~ -print0 \)|
       cpio -pmd0 /dest-dir

       This command copies the contents of /source-dir to /dest-dir, but omits
       files  and directories named .snapshot (and anything in them).  It also
       omits files or directories whose name ends in ~,  but  not  their  con‐
       tents.  The construct -prune -o \( ... -print0 \) is quite common.  The
       idea here is that the expression before -prune matches things which are
       to  be  pruned.  However, the -prune action itself returns true, so the
       following -o ensures that the right hand side  is  evaluated  only  for
       those  directories  which didn't get pruned (the contents of the pruned
       directories are not even visited, so their  contents  are  irrelevant).
       The  expression on the right hand side of the -o is in parentheses only
       for clarity.  It emphasises that the -print0 action  takes  place  only
       for  things  that  didn't  have  -prune  applied  to them.  Because the
       default `and' condition between tests binds more tightly than -o,  this
       is  the  default anyway, but the parentheses help to show what is going
       on.

Answer 4

Quick Start

Run:

rsync -av --exclude='path1/in/source' --exclude='path2/in/source' [source]/ [destination]

---

Notes

• -avr will create a new directory named [destination].
• source and source/ create different results:
  • source — copy the contents of source into destination.
  • source/ — copy the folder source into destination.
• To exclude many files:
  • --exclude-from=FILE — FILE is the name of a file containing other files or directories to exclude.
• --exclude may also contain wildcards:
  • e.g. --exclude=*/.svn*

Modified from: https://stackoverflow.com/a/2194500/749232

---

Example

Starting folder structure:

.
├── destination
└── source
    ├── fileToCopy.rtf
    └── fileToExclude.rtf

Run:

rsync -av --exclude='fileToCopy.rtf' source/ destination

Ending folder structure:

.
├── destination
│   └── fileToExclude.rtf
└── source
    ├── fileToCopy.rtf
    └── fileToExclude.rtf

Answer 5

you can use tar, with --exclude option , and then untar it in destination. eg

cd /source_directory
tar cvf test.tar --exclude=dir_to_exclude *
mv test.tar /destination 
cd /destination  
tar xvf test.tar

see the man page of tar for more info

Answer 6

Similar to Jeff's idea (untested):

find . -name * -print0 | grep -v "exclude" | xargs -0 -I {} cp -a {} destination/

Answer 7

Simple solution (but I would still prefer the bash pattern matching from the top comments):

touch /path/to/target/.git
cp -n -ax * /path/to/target/
rm /path/to/target/.git

This exploits the -n option of cp, which forces cp to not overwrite existing targets.

Drawback: Works with GNU cp. If you don't have GNU cp, then the cp operation might return an error code (1), which is annoying because then you can't tell if it was a real failure.

Answer 8

inspired by @SteveLazaridis's answer, which would fail, here is a POSIX shell function - just copy and paste into a file named cpx in yout $PATH and make it executible (chmod a+x cpr). [Source is now maintained in my GitLab.

#!/bin/sh

# usage: cpx [-n|--dry-run] "from_path" "to_path" "newline_separated_exclude_list"
# limitations: only excludes from "from_path", not it's subdirectories

cpx() {
# run in subshell to avoid collisions
  (_CopyWithExclude "$@")
}

_CopyWithExclude() {
  case "$1" in
    -n|--dry-run) { DryRun='echo'; shift; } ;;
  esac

  from="$1"
  to="$2"
  exclude="$3"

  $DryRun mkdir -p "$to"

  if [ -z "$exclude" ]; then
      cp "$from" "$to"
      return
  fi

  ls -A1 "$from" \
    | while IFS= read -r f; do
        unset excluded
        if [ -n "$exclude" ]; then
          for x in $(printf "$exclude"); do
          if [ "$f" = "$x" ]; then
              excluded=1
              break
          fi
          done
        fi
        f="${f#$from/}"
        if [ -z "$excluded" ]; then
          $DryRun cp -R "$f" "$to"
        else
          [ -n "$DryRun" ] && echo "skip '$f'"
        fi
      done
}

# Do not execute if being sourced
[ "${0#*cpx}" != "$0" ] && cpx "$@"

Example usage

EXCLUDE="
.git
my_secret_stuff
"
cpr "$HOME/my_stuff" "/media/usb" "$EXCLUDE"

Answer 9

EXCLUDE="foo bar blah jah"                                                                             
DEST=$1

for i in *
do
    for x in $EXCLUDE
    do  
        if [ $x != $i ]; then
            cp -a $i $DEST
        fi  
    done
done

Untested...