1075

Is there a simple shell command/script that supports excluding certain files/folders from being archived?

I have a directory that need to be archived with a sub directory that has a number of very large files I do not need to backup.

Not quite solutions:

The tar --exclude=PATTERN command matches the given pattern and excludes those files, but I need specific files & folders to be ignored (full file path), otherwise valid files might be excluded.

I could also use the find command to create a list of files and exclude the ones I don't want to archive and pass the list to tar, but that only works with for a small amount of files. I have tens of thousands.

I'm beginning to think the only solution is to create a file with a list of files/folders to be excluded, then use rsync with --exclude-from=file to copy all the files to a tmp directory, and then use tar to archive that directory.

Can anybody think of a better/more efficient solution?

EDIT: Charles Ma's solution works well. The big gotcha is that the --exclude='./folder' MUST be at the beginning of the tar command. Full command (cd first, so backup is relative to that directory):

cd /folder_to_backup
tar --exclude='./folder' --exclude='./upload/folder2' -zcvf /backup/filename.tgz .
Nathan majicvr.com
  • 950
  • 2
  • 11
  • 31
deepwell
  • 20,195
  • 10
  • 33
  • 39
  • 170
    Another thing caught me out on that, might be worth a note: **Trailing slashes** at the end of excluded folders will cause tar to **not** exclude those folders at all. – Rekhyt May 01 '12 at 12:55
  • 7
    @Rekhyt thanks, I was staring at the command for 15 minutes ... then 30 – earcam Jun 06 '12 at 22:07
  • 12
    It seems the position of --exclude depends on the version of tar. For tar 1.23, --exclude needs to come after the main commands. – Joel G Mathew Mar 14 '13 at 06:19
  • 3
    Don't forget the "'" (quotation marks). – Meetai.com Dec 07 '13 at 06:20
  • 2
    I had to remove the single quotation marks in order to exclude sucessfully the directories. (`tar -zcvf gatling-charts-highcharts-1.4.6.tar.gz /opt/gatling-charts-highcharts-1.4.6 --exclude=results --exclude=target`) – bric3 Jun 24 '14 at 16:06
  • It only worked without the quotation marks for me as well – guidod Sep 30 '15 at 18:46
  • 1
    The `--exclude='./folder'` syntax does not seem to work on OS X. – staffan Oct 20 '15 at 13:22
  • 1
    it works with quotes i use this one `tar --exclude "logs" --exclude "*.tar.gz" -zcvf "Archive.tar.gz" -C "/path/to/files" .` The `--exclude` must go first i had problems until i did that. Here i'm using the -C option; change to directory. That helps make it insensitive to the initial directory TAR is running from. – escape-llc Nov 06 '15 at 11:01
  • 7
    `--exclude` doesn't have to be first but it has to come somewhere before the source directory (tested with tar 1.29 on Cygwin). – wortwart Sep 12 '17 at 23:50
  • In my case I needed also to remove the initial './' characters inside the "-exclude" options... but I think it depends on what you have specified as the last parameter (tar version "tar (GNU tar) 1.26"), for example: `tar --exclude='wlserver_12.2/OPatch/patches' --exclude='wlserver_12.2/OPatch_20191007/patches' -cvf wlserver_12.2.backup.tar wlserver_12.2` – Mariano Paniga Mar 08 '22 at 09:57
  • You can use the `*` for the pattern to match subfolders. But then running from *bash* and other shells the asterisk needs escaping: Use `--exclude=/folder-from-root/\*` when running from bash to exclude the folder from root plus subfolders and all files in those – miiimooo Jun 26 '23 at 11:07

30 Answers30

1388

You can have multiple exclude options for tar so

$ tar --exclude='./folder' --exclude='./upload/folder2' -zcvf /backup/filename.tgz .

etc will work. Make sure to put --exclude before the source and destination items.

ericosg
  • 4,926
  • 4
  • 37
  • 59
Charles Ma
  • 47,141
  • 22
  • 87
  • 101
  • Can you give an example of a pattern. Is this a regex pattern? – hitautodestruct Jun 29 '14 at 07:44
  • No. It's not a RegExp. The `*` here means any character (including `/`). – Nux Jul 13 '14 at 20:05
  • 100
    This answer makes it look like --exclude comes first... tar cvfpz ../stuff.tgz --exclude='node_modules' --exclude='.git' . – James O'Brien Oct 17 '14 at 03:37
  • 1
    See @Stephen Donecker answer below to have a file containing the list of files to exclude. – user276648 Nov 20 '14 at 01:45
  • 17
    As an example, if you are trying to backup your wordpress project folder, excluding the uploads folder, you can use this command: **tar -cvf wordpress_backup.tar wordpress --exclude=wp-content/uploads** – shasi kanth Feb 27 '15 at 10:49
  • 21
    I came up with the following command: `tar -zcv --exclude='file1' --exclude='patter*' --exclude='file2' -f /backup/filename.tgz .` note that the `-f` flag needs to precede the tar file see: http://superuser.com/a/559341/415047 – Alfred Bez Jul 16 '15 at 07:28
  • 26
    A "/" on the end of the exclude directory will cause it to fail. I guess tar thinks an ending / is part of the directory name to exclude. BAD: --exclude=mydir/ GOOD: --exclude=mydir – Josiah Aug 21 '15 at 16:22
  • 1
    @hitautodestruct The patterns are called file globs, see http://man7.org/linux/man-pages/man7/glob.7.html for documentation – Rik Smith-Unna Sep 14 '15 at 11:10
  • 8
    > Make sure to put --exclude before the source and destination items. OR use an absolute path for the exclude: tar -cvpzf backups/target.tar.gz --exclude='/home/username/backups' /home/username – NightKnight on Cloudinsidr.com Nov 24 '16 at 09:55
  • 2
    the ordering of the exclude tag matters. But how is this behaviour not considered a bug? Would be an easy fix. – gdbj Jan 15 '17 at 21:38
  • I'm guessing bash expansion should work for exclude, `--exclude={'folder1','folder2','folder3'}` , saves from having to type too many excludes –  Jan 24 '17 at 13:54
  • I suggest this one for multi-core machines : `tar --exclude='./folder' --exclude='./upload/folder2' -c --use-compress-program=pigz -f /backup/filename.tgz .` – manuc66 Feb 16 '17 at 12:14
  • 1
    i'm 100% sure everyone doubled back to this issue after *not* having put `--exclude=` before the rest. lets make that bold? – ericosg May 16 '17 at 08:14
  • 1
    As of tar 1.28, at least, the ordering does not appear to matter. `tar -czf .tar.gz --exclude={*.mp4,.git} ` also works. – Brent Faust Oct 26 '18 at 21:07
  • 2
    Note that the path to the directory to exclude shouldn't end with a slash. ` --exclude='./folder'` works, but --exclude='./folder/' doesn't work. – Paul Sep 23 '19 at 17:38
  • 1
    Tar complexity! [xkcd: tar](https://xkcd.com/1168/) – F. Hauri - Give Up GitHub Nov 21 '20 at 07:50
  • This is not the answer. I tried putting the `--exclude-from` first and I tried providing absolute paths to exclude. Either nothing was excluded, or the excluded patterns also matched in subdirectories, excluding too much! – ygoe Dec 03 '20 at 19:35
  • can I exclude hidden folders as well? – alper Feb 10 '21 at 14:57
  • It looks like there's an `--anchored` flag for preventing it from matching the middle of paths. Apparently the intended use case is `--exclude="*.o"` and you have to fight to get anything else to work. – sfink Jul 23 '21 at 17:27
  • On a Mac as of at least 11.6 (with bsdtar 3.3.2) `--exclude` must precede `-zcvf`. For example: `tar --exclude "./.git" --exclude "./node_modules" -zcvf archive.tar.gz some_directory`. (Update: I just noticed @Jerinaw answer, that also works. Looks like `-f` is the problem and must be after the `--exclude` options) – wsams Oct 25 '21 at 23:52
  • Does not work, it still includes the excluded folder – Black Nov 04 '21 at 10:09
  • It's pretty funny that `-zcvf` works while `-cvfz` just creates an archive named `z` and gg to everyone :D – Avio May 04 '22 at 14:40
  • quotation marks do not work for my case!! `--exclude=src/xyz/target` – Russo Jan 10 '23 at 08:45
  • It's important for the format of the `--excludes` argument to match the name used for the tar. For example, if you say `tar -cvf foo.tar foo-dir`, then you need to add `--excludes=foo-dir/subdir1` or whatever sub-directory you want to exclude. If you use the command `tar -cvf foo.tar ./foo-dir` then you need to include the `./` in the excludes argument such as `--excludes=./foo-dir/subdir1`. Similarly if you say `tar cvf foo.tar .` then you need to use `./subdir1` as `subdir1` while it's the same directory won't match (and won't be excluded) – PatS Jun 15 '23 at 16:15
  • nightknight's comment to include the fully qualified absolute path is the only way I could get the --exclude to work. thanks! – Jim Aug 07 '23 at 16:03
154

You can exclude directories with --exclude for tar.

If you want to archive everything except /usr you can use:

tar -zcvf /all.tgz / --exclude=/usr

In your case perhaps something like

tar -zcvf archive.tgz arc_dir --exclude=dir/ignore_this_dir
fedorqui
  • 275,237
  • 103
  • 548
  • 598
Johan Soderberg
  • 2,650
  • 1
  • 15
  • 12
87

Possible options to exclude files/directories from backup using tar:

Exclude files using multiple patterns

tar -czf backup.tar.gz --exclude=PATTERN1 --exclude=PATTERN2 ... /path/to/backup

Exclude files using an exclude file filled with a list of patterns

tar -czf backup.tar.gz -X /path/to/exclude.txt /path/to/backup

Exclude files using tags by placing a tag file in any directory that should be skipped

tar -czf backup.tar.gz --exclude-tag-all=exclude.tag /path/to/backup
Stephen Donecker
  • 2,361
  • 18
  • 8
64

old question with many answers, but I found that none were quite clear enough for me, so I would like to add my try.

if you have the following structure

/home/ftp/mysite/

with following file/folders

/home/ftp/mysite/file1
/home/ftp/mysite/file2
/home/ftp/mysite/file3
/home/ftp/mysite/folder1
/home/ftp/mysite/folder2
/home/ftp/mysite/folder3

so, you want to make a tar file that contain everyting inside /home/ftp/mysite (to move the site to a new server), but file3 is just junk, and everything in folder3 is also not needed, so we will skip those two.

we use the format

tar -czvf <name of tar file> <what to tar> <any excludes>

where the c = create, z = zip, and v = verbose (you can see the files as they are entered, usefull to make sure none of the files you exclude are being added). and f= file.

so, my command would look like this

cd /home/ftp/
tar -czvf mysite.tar.gz mysite --exclude='file3' --exclude='folder3'

note the files/folders excluded are relatively to the root of your tar (I have tried full path here relative to / but I can not make that work).

hope this will help someone (and me next time I google it)

Sverre
  • 1,318
  • 13
  • 21
  • I would have added a comment on some of the other good answers, but did not have the karma, so decided to make a full answer. – Sverre May 08 '14 at 10:53
  • 7
    This answer definitely helped me! The gotcha for me was that my command looked something like `tar -czvf mysite.tar.gz mysite --exclude='./mysite/file3' --exclude='./mysite/folder3'`, and this didn't exclude anything. – Anish Ramaswamy May 16 '15 at 00:11
  • 2
    Your sample was very similar to what I had issue with! Thank you! – Qorbani Dec 02 '16 at 06:14
  • 3
    Nice and clear thank you. For me the issue was that other answers include absolute of relative paths. But all you have to do is add the name of the folder you want to exclude. – Hubert Feb 22 '17 at 07:38
  • 2
    This is a much more clear answer. Because of the example I was able to get it working as the paths were confusing at first. Thanks a bunch! – fagiani Jul 19 '18 at 13:35
  • 2
    it worked! Please remember not to add a trailing slash to the exclude. For example, while "file3" or "file3/subfolder" works, "file3/" and "file3/subfolder/" do not! – lucaferrario Apr 01 '19 at 16:18
  • 1
    For me this worked when I did not surround the exclude files and directories with quotation marks. – SherylHohman May 19 '19 at 01:53
43

You can use standard "ant notation" to exclude directories relative.
This works for me and excludes any .git or node_module directories:

tar -cvf myFile.tar --exclude=**/.git/* --exclude=**/node_modules/*  -T /data/txt/myInputFile.txt 2> /data/txt/myTarLogFile.txt

myInputFile.txt contains:

/dev2/java
/dev2/javascript

Ronan Boiteau
  • 9,608
  • 6
  • 34
  • 56
Eric Manley
  • 1,049
  • 10
  • 8
  • 5
    I believe this require that the Bash shell option variable `globstar` has to be enabled. Check with `shopt -s globstar`. I think it **off** by default on most unix based OS's. From Bash manual: "**globstar:** *If set, the pattern `**` used in a filename expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a ‘/’, only directories and subdirectories match.*" – not2qubit Apr 04 '18 at 03:24
  • Worked for me. The mistake I was making was that I was putting the --exclude's after the source. Once I changed it to put them before the source it worked fine. – Jose Quijada Aug 19 '22 at 00:14
  • There's an option in the `tar` itself for the job you're doing: `--exclude-vcs`. From `tar(1)`: "Exclude version control system directories." – mathway Oct 02 '22 at 00:02
  • 2
    this didn't work for me on MacOS, but `--exclude=.git --exclude=node_modules` worked. i also had to make the `--exlcude` params the first ones after `tar` (i.e. before the `-c`. i think that was because of some other options i was using, but thought i'd leave it here for anyone else's benefit – Kip Jan 28 '23 at 16:39
26

This exclude pattern handles filename suffix like png or mp3 as well as directory names like .git and node_modules

tar --exclude={*.png,*.mp3,*.wav,.git,node_modules} -Jcf ${target_tarball}  ${source_dirname}
Scott Stensland
  • 26,870
  • 12
  • 93
  • 104
18

I've experienced that, at least with the Cygwin version of tar I'm using ("CYGWIN_NT-5.1 1.7.17(0.262/5/3) 2012-10-19 14:39 i686 Cygwin" on a Windows XP Home Edition SP3 machine), the order of options is important.

While this construction worked for me:

tar cfvz target.tgz --exclude='<dir1>' --exclude='<dir2>' target_dir

that one didn't work:

tar cfvz --exclude='<dir1>' --exclude='<dir2>' target.tgz target_dir

This, while tar --help reveals the following:

tar [OPTION...] [FILE]

So, the second command should also work, but apparently it doesn't seem to be the case...

Best rgds,

GeertVc
  • 1,914
  • 17
  • 22
  • 6
    Just want to add to the above, that it is important that the directory to be excluded should NOT contain a final backslash. So, **--exclude='/path/to/exclude/dir'** is _CORRECT_, **--exclude='/path/to/exclude/dir/'** is _WRONG_. – GeertVc Dec 31 '13 at 13:35
  • 21
    this is because the target archive target.tgz is an argument of the f switch, which it should follow – Valentino Feb 09 '15 at 13:37
17

After reading all this good answers for different versions and having solved the problem for myself, I think there are very small details that are very important, and rare to GNU/Linux general use, that aren't stressed enough and deserves more than comments.

So I'm not going to try to answer the question for every case, but instead, try to register where to look when things doesn't work.

IT IS VERY IMPORTANT TO NOTICE:

  1. THE ORDER OF THE OPTIONS MATTER: it is not the same put the --exclude before than after the file option and directories to backup. This is unexpected at least to me, because in my experience, in GNU/Linux commands, usually the order of the options doesn't matter.
  2. Different tar versions expects this options in different order: for instance, @Andrew's answer indicates that in GNU tar v 1.26 and 1.28 the excludes comes last, whereas in my case, with GNU tar 1.29, it's the other way.
  3. THE TRAILING SLASHES MATTER: at least in GNU tar 1.29, it shouldn't be any.

In my case, for GNU tar 1.29 on Debian stretch, the command that worked was

tar --exclude="/home/user/.config/chromium" --exclude="/home/user/.cache" -cf file.tar  /dir1/ /home/ /dir3/

The quotes didn't matter, it worked with or without them.

I hope this will be useful to someone.

user2553863
  • 682
  • 1
  • 8
  • 17
  • 2
    Thank you for your answer. I was looking (what felt like a very long time) for a solution, and your answer guided me in the right direction. However, in my case (Ubuntu 18.04.3, Tar 1.29) I only could make it work with adding the folder name and NOT the path, e.g.: tar --exclude=folder1 --exclude=folder2 -czvf /opt/archieve.tgz folder – Nitai Oct 10 '19 at 22:04
  • I tried most options on this page and on tar version 1.27.1, this answer helped me. – mattlangtree Feb 02 '22 at 11:57
17

I'd like to show another option I used to get the same result as the answers before provide, I had a similar case where I wanted to backup android studio projects all together in a tar file to upload to media fire, using the du command to find the large files, I found that I didn't need some directories like: build, linux e .dart_tools Using the first answer of Charles_ma I modified it a little bit to be able to run the command from the parent directory of the my Android directory.

tar --exclude='*/build' --exclude='*/linux' --exclude='*/.dart_tool' -zcvf androidProjects.tar Android/

It worked like a charm.

Ps. Sorry if this kind of answer is not allowed, if this is the case I will remove.

Rck
  • 326
  • 2
  • 5
15

If you are trying to exclude Version Control System (VCS) files, tar already supports two interesting options about it! :)

  1. Option : --exclude-vcs

This option excludes files and directories used by following version control systems: CVS, RCS, SCCS, SVN, Arch, Bazaar, Mercurial, and Darcs.

As of version 1.32, the following files are excluded:

  • CVS/, and everything under it
  • RCS/, and everything under it
  • SCCS/, and everything under it
  • .git/, and everything under it
  • .gitignore
  • .gitmodules
  • .gitattributes
  • .cvsignore
  • .svn/, and everything under it
  • .arch-ids/, and everything under it
  • {arch}/, and everything under it
  • =RELEASE-ID
  • =meta-update
  • =update
  • .bzr
  • .bzrignore
  • .bzrtags
  • .hg
  • .hgignore
  • .hgrags

  • _darcs

    1. Option : --exclude-vcs-ignores

When archiving directories that are under some version control system (VCS), it is often convenient to read exclusion patterns from this VCS' ignore files (e.g. .cvsignore, .gitignore, etc.) This option provide such possibility.

Before archiving a directory, see if it contains any of the following files: cvsignore, .gitignore, .bzrignore, or .hgignore. If so, read ignore patterns from these files.

The patterns are treated much as the corresponding VCS would treat them, i.e.:

.cvsignore

Contains shell-style globbing patterns that apply only to the directory where this file resides. No comments are allowed in the file. Empty lines are ignored.

.gitignore

Contains shell-style globbing patterns. Applies to the directory where .gitfile is located and all its subdirectories.

Any line beginning with a # is a comment. Backslash escapes the comment character.

.bzrignore

Contains shell globbing-patterns and regular expressions (if prefixed with RE:(16). Patterns affect the directory and all its subdirectories.

Any line beginning with a # is a comment.

.hgignore

Contains posix regular expressions(17). The line syntax: glob switches to shell globbing patterns. The line syntax: regexp switches back. Comments begin with a #. Patterns affect the directory and all its subdirectories.

  1. Example

tar -czv --exclude-vcs --exclude-vcs-ignores -f path/to/my-tar-file.tar.gz path/to/my/project/

T.M.
  • 589
  • 8
  • 10
14

I found this somewhere else so I won't take credit, but it worked better than any of the solutions above for my mac specific issues (even though this is closed):

tar zc --exclude __MACOSX --exclude .DS_Store -f <archive> <source(s)>
Rob
  • 4,093
  • 5
  • 44
  • 54
  • 2
    Thanks for this answer, the tar on darwin definitely has a different syntax and it was driving me nuts why "--exclude=blah" in the other answers weren't working. This worked great on a mac. – Michael Sep 16 '15 at 14:14
  • 1
    Don't forget `COPYFILE_DISABLE=1` when using tar, otherwise [you may get ._ files in your tarball](http://superuser.com/q/61185/62656) – Benoit Duffez Jun 19 '16 at 21:14
  • Thanks for including your answer. It's always nice to include a link to the source where you found the answer. Bonus: if the source was from another stackoverflow or stackexchange post, you get extra karma (either points or badges - I don't remember which). Either way, they get a smile, and everyone wins. No downsides :-) It also helps people if who want to search out extra info. Sometimes people will upvote just because you included a source link. Finally, sharing the specific issue this addressed, or why this was a better solution, it might help someone else with a unique problem. – SherylHohman May 18 '19 at 23:29
13

For Mac OSX I had to do

tar -zcv --exclude='folder' -f theOutputTarFile.tar folderToTar

Note the -f after the --exclude=

Jerinaw
  • 5,260
  • 7
  • 41
  • 54
11

For those who have issues with it, some versions of tar would only work properly without the './' in the exclude value.

Tar --version

tar (GNU tar) 1.27.1

Command syntax that work:

tar -czvf ../allfiles-butsome.tar.gz * --exclude=acme/foo

These will not work:

$ tar -czvf ../allfiles-butsome.tar.gz * --exclude=./acme/foo
$ tar -czvf ../allfiles-butsome.tar.gz * --exclude='./acme/foo'
$ tar --exclude=./acme/foo -czvf ../allfiles-butsome.tar.gz *
$ tar --exclude='./acme/foo' -czvf ../allfiles-butsome.tar.gz *
$ tar -czvf ../allfiles-butsome.tar.gz * --exclude=/full/path/acme/foo
$ tar -czvf ../allfiles-butsome.tar.gz * --exclude='/full/path/acme/foo'
$ tar --exclude=/full/path/acme/foo -czvf ../allfiles-butsome.tar.gz *
$ tar --exclude='/full/path/acme/foo' -czvf ../allfiles-butsome.tar.gz *
Samuel Liew
  • 76,741
  • 107
  • 159
  • 260
J. Lawson
  • 123
  • 1
  • 8
9

I agree the --exclude flag is the right approach.

$ tar --exclude='./folder_or_file' --exclude='file_pattern' --exclude='fileA'

A word of warning for a side effect that I did not find immediately obvious: The exclusion of 'fileA' in this example will search for 'fileA' RECURSIVELY!

Example:A directory with a single subdirectory containing a file of the same name (data.txt)

data.txt
config.txt
--+dirA
  |  data.txt
  |  config.docx

  • If using --exclude='data.txt' the archive will not contain EITHER data.txt file. This can cause unexpected results if archiving third party libraries, such as a node_modules directory.

  • To avoid this issue make sure to give the entire path, like --exclude='./dirA/data.txt'

Aaron Votre
  • 1,620
  • 1
  • 12
  • 8
6

After reading this thread, I did a little testing on RHEL 5 and here are my results for tarring up the abc directory:

This will exclude the directories error and logs and all files under the directories:

tar cvpzf abc.tgz abc/ --exclude='abc/error' --exclude='abc/logs'

Adding a wildcard after the excluded directory will exclude the files but preserve the directories:

tar cvpzf abc.tgz abc/ --exclude='abc/error/*' --exclude='abc/logs/*'
Undo
  • 25,519
  • 37
  • 106
  • 129
Mike
  • 61
  • 1
  • 1
  • In the second example above there should be asterisks after the last slash in each exclude clause, but the post did not take them. – Mike May 09 '14 at 21:29
5

To avoid possible 'xargs: Argument list too long' errors due to the use of find ... | xargs ... when processing tens of thousands of files, you can pipe the output of find directly to tar using find ... -print0 | tar --null ....

# archive a given directory, but exclude various files & directories 
# specified by their full file paths
find "$(pwd -P)" -type d \( -path '/path/to/dir1' -or -path '/path/to/dir2' \) -prune \
   -or -not \( -path '/path/to/file1' -or -path '/path/to/file2' \) -print0 | 
   gnutar --null --no-recursion -czf archive.tar.gz --files-from -
   #bsdtar --null -n -czf archive.tar.gz -T -
carlo
  • 51
  • 1
  • 1
  • you can quote 'exclude' string, like this: 'somedir/filesdir/*' then shell isn't going to expand asterisks and other white chars. – Znik Mar 04 '14 at 12:20
  • `xargs -n 1` is another option to avoid `xargs: Argument list too long` error ;) – Tuxdude Nov 15 '14 at 05:12
4

You can also use one of the "--exclude-tag" options depending on your needs:

  • --exclude-tag=FILE
  • --exclude-tag-all=FILE
  • --exclude-tag-under=FILE

The folder hosting the specified FILE will be excluded.

frommelmak
  • 41
  • 1
3

gnu tar v 1.26 the --exclude needs to come after archive file and backup directory arguments, should have no leading or trailing slashes, and prefers no quotes (single or double). So relative to the PARENT directory to be backed up, it's:

tar cvfz /path_to/mytar.tgz ./dir_to_backup --exclude=some_path/to_exclude

Andrew
  • 906
  • 7
  • 9
3

Use the find command in conjunction with the tar append (-r) option. This way you can add files to an existing tar in a single step, instead of a two pass solution (create list of files, create tar).

find /dir/dir -prune ... -o etc etc.... -exec tar rvf ~/tarfile.tar {} \;
Alex B
  • 24,678
  • 14
  • 64
  • 87
3

You can use cpio(1) to create tar files. cpio takes the files to archive on stdin, so if you've already figured out the find command you want to use to select the files the archive, pipe it into cpio to create the tar file:

find ... | cpio -o -H ustar | gzip -c > archive.tar.gz
camh
  • 40,988
  • 13
  • 62
  • 70
2
tar -cvzf destination_folder source_folder -X /home/folder/excludes.txt

-X indicates a file which contains a list of filenames which must be excluded from the backup. For Instance, you can specify *~ in this file to not include any filenames ending with ~ in the backup.

Ashwini Gupta
  • 88
  • 7
  • I think, this is the best solution, as it even works in those cases, that the number of excludes is large. It is also possible to include the X option in the option pack, so the shortest form is probably: `tar cXvfJ EXCLUDE-LIST ARCHIVE.tar.xz SOURCE-FOLDER` – Kai Petzke Jan 02 '20 at 23:13
2

Success Case: 1) if giving full path to take backup, in exclude also should be used full path.

tar -zcvf /opt/ABC/BKP_27032020/backup_27032020.tar.gz --exclude='/opt/ABC/csv/' --exclude='/opt/ABC/log/' /opt/ABC

2) if giving current path to take backup, in exclude also should be used current path only.

tar -zcvf backup_27032020.tar.gz --exclude='ABC/csv/' --exclude='ABC/log/' ABC

Failure Case:

  1. if giving currentpath directory to take backup and full path to ignore,then wont work

    tar -zcvf /opt/ABC/BKP_27032020/backup_27032020.tar.gz --exclude='/opt/ABC/csv/' --exclude='/opt/ABC/log/' ABC

Note: mentioning exclude before/after backup directory is fine.

Sridhar Kumar N
  • 61
  • 2
2

It seems to be impossible to exclude directories with absolute paths. As soon as ANY of the paths are absolute (source or/and exclude) the exclude command will not work. That's my experience after trying all possible combinations.

ESP32
  • 8,089
  • 2
  • 40
  • 61
1

Check it out

tar cvpzf zip_folder.tgz . --exclude=./public --exclude=./tmp --exclude=./log --exclude=fileName
RohitPorwal
  • 1,045
  • 15
  • 23
1

I want to have fresh front-end version (angular folder) on localhost. Also, git folder is huge in my case, and I want to exclude it. I need to download it from server, and unpack it in order to run application.

Compress angular folder from /var/lib/tomcat7/webapps, move it to /tmp folder with name angular.23.12.19.tar.gz

Command : 

tar --exclude='.git' -zcvf /tmp/angular.23.12.19.tar.gz /var/lib/tomcat7/webapps/angular/
dobrivoje
  • 848
  • 1
  • 9
  • 18
0

Possible redundant answer but since I found it useful, here it is:

While a FreeBSD root (i.e. using csh) I wanted to copy my whole root filesystem to /mnt but without /usr and (obviously) /mnt. This is what worked (I am at /):

tar --exclude ./usr --exclude ./mnt --create --file - . (cd /mnt && tar xvd -)

My whole point is that it was necessary (by putting the ./) to specify to tar that the excluded directories where part of the greater directory being copied.

My €0.02

George
  • 161
  • 6
0

I had no luck getting tar to exclude a 5 Gigabyte subdirectory a few levels deep. In the end, I just used the unix Zip command. It worked a lot easier for me.

So for this particular example from the original post
(tar --exclude='./folder' --exclude='./upload/folder2' -zcvf /backup/filename.tgz . )

The equivalent would be:

zip -r /backup/filename.zip . -x upload/folder/**\* upload/folder2/**\*

(NOTE: Here is the post I originally used that helped me https://superuser.com/questions/312301/unix-zip-directory-but-excluded-specific-subdirectories-and-everything-within-t)

Community
  • 1
  • 1
user2792605
  • 1
  • 1
0

I've never made tar --exclude option work for me. In my case using rsync to copy folder tree to new location and then using standard tar worked.

rsync -av --exclude='node_modules' --exclude='.git' folder_tree folder_tree_excluded
tar -cvzf archive.tar.gz folder_tree_excluded
Tom Raganowicz
  • 2,169
  • 5
  • 27
  • 41
0

Your best bet is to use find with tar, via xargs (to handle the large number of arguments). For example:

find / -print0 | xargs -0 tar cjf tarfile.tar.bz2
Joe
  • 818
  • 6
  • 7
  • 4
    That can cause tar to be invoked multiple times - and will also pack files repeatedly. Correct is: `find / -print0 | tar -T- --null --no-recursive -cjf tarfile.tar.bz2` – jørgensen Mar 04 '12 at 15:23
  • I read somewhere that when using `xargs`, one should use tar `r` option instead of `c` because when `find` actually finds loads of results, the xargs will split those results (based on the local command line arguments limit) into chuncks and invoke tar on each part. This will result in a archive containing the last chunck returned by `xargs` and not all results found by the `find` command. – Stphane Dec 19 '15 at 11:10
-1

The following bash script should do the trick. It uses the answer given here by Marcus Sundman.

#!/bin/bash

echo -n "Please enter the name of the tar file you wish to create with out extension "
read nam

echo -n "Please enter the path to the directories to tar "
read pathin

echo tar -czvf $nam.tar.gz
excludes=`find $pathin -iname "*.CC" -exec echo "--exclude \'{}\'" \;|xargs`
echo $pathin

echo tar -czvf $nam.tar.gz $excludes $pathin

This will print out the command you need and you can just copy and paste it back in. There is probably a more elegant way to provide it directly to the command line.

Just change *.CC for any other common extension, file name or regex you want to exclude and this should still work.

EDIT

Just to add a little explanation; find generates a list of files matching the chosen regex (in this case *.CC). This list is passed via xargs to the echo command. This prints --exclude 'one entry from the list'. The slashes () are escape characters for the ' marks.

Community
  • 1
  • 1
James
  • 223
  • 1
  • 3
  • 9
  • 1
    Requiring interactive input is a poor design choice for most shell scripts. Make it read command-line parameters instead and you get the benefit of the shell's tab completion, history completion, history editing, etc. – tripleee Sep 14 '17 at 04:27
  • 1
    Additionally, your script does not work for paths which contain whitespace or shell metacharacters. You should *basically always* put variables in double quotes unless you specifically require the shell to perform whitespace tokenization and wildcard expansion. For details, please see https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable – tripleee Sep 14 '17 at 04:38