tar: add all files and directories in current directory INCLUDING .svn and so on

Question

I try to tar.gz a directory and use

tar -czf workspace.tar.gz *

The resulting tar includes .svn directories in subdirs but NOT in the current directory (as * gets expanded to only 'visible' files before it is passed to tar

I tried to

tar -czf workspace.tar.gz . instead but then I am getting an error because '.' has changed while reading:

tar: ./workspace.tar.gz: file changed as we read it

Is there a trick so that * matches all files (including dot-prefixed) in a directory?

(using bash on Linux SLES-11 (2.6.27.19)

Maybe this question belongs on superuser? – Peter Jaric Sep 06 '10 at 13:44 — Peter Jaric, Sep 06 '10 at 13:44

Jonathan Leffler · Answer 1 · 2010-09-06T18:42:21.977

Don't create the tar file in the directory you are packing up:

tar -czf /tmp/workspace.tar.gz .

does the trick, except it will extract the files all over the current directory when you unpack. Better to do:

cd ..
tar -czf workspace.tar.gz workspace

or, if you don't know the name of the directory you were in:

base=$(basename $PWD)
cd ..
tar -czf $base.tar.gz $base

(This assumes that you didn't follow symlinks to get to where you are and that the shell doesn't try to second guess you by jumping backwards through a symlink - bash is not trustworthy in this respect. If you have to worry about that, use cd -P .. to do a physical change directory. Stupid that it is not the default behaviour in my view - confusing, at least, for those for whom cd .. never had any alternative meaning.)

One comment in the discussion says:

I [...] need to exclude the top directory and I [...] need to place the tar in the base directory.

The first part of the comment does not make much sense - if the tar file contains the current directory, it won't be created when you extract file from that archive because, by definition, the current directory already exists (except in very weird circumstances).

The second part of the comment can be dealt with in one of two ways:

Either: create the file somewhere else - /tmp is one possible location - and then move it back to the original location after it is complete.
Or: if you are using GNU Tar, use the --exclude=workspace.tar.gz option. The string after the = is a pattern - the example is the simplest pattern - an exact match. You might need to specify --exclude=./workspace.tar.gz if you are working in the current directory contrary to recommendations; you might need to specify --exclude=workspace/workspace.tar.gz if you are working up one level as suggested. If you have multiple tar files to exclude, use '*', as in --exclude=./*.gz.

Good answer but I've got a situation where I'm forced to run this command inside the directory that has to be packed. I have insufficiend permissions to run it outside. — Hexodus, Jun 09 '14 at 11:59
@Hexodus: you can run the command from within the current directory if need be, but you need to ensure that `tar` does not try to archive the file it is creating. There are multiple ways to achieve that, especially with [GNU Tar]( https://www.gnu.org/software/tar/manual/tar.html). The simplest is to create the tar file in another directory — `/tmp` for example. If you can't create files outside the directory you're archiving, then use the `--exclude="name-of-tar-file"` option. If you're using an alternative (non-GNU) version of `tar`, read its manual. — Jonathan Leffler, Nov 16 '21 at 16:58
Using `cd ..` changes your current directory. If you want to go back (after the tar command), use the `cd -` and you are again in the same directory as before. — Thomas, Feb 18 '22 at 12:52

score 88 · Answer 2 · answered Sep 19 '16 at 09:21

88

There are a couple of steps to take:

Replace * by . to include hidden files as well.
To create the archive in the same directory a --exclude=workspace.tar.gz can be used to exclude the archive itself.
To prevent the tar: .: file changed as we read it error when the archive is not yet created, make sure it exists (e.g. using touch), so the --exclude matches with the archive filename. (It does not match it the file does not exists)

Combined this results in the following script:

touch workspace.tar.gz
tar -czf workspace.tar.gz --exclude=workspace.tar.gz .

answered Sep 19 '16 at 09:21

Veger

37,240
11
105
116

11

I read all answers and comments and this is the nicest way to create tarball in the same dir. From this answer I also understood why this error happens. – Szymon Sadło Dec 29 '17 at 16:48
If you want to see the progress while it's compressing your tar file, you can add an additional flag for `--verbose`, like so: `-czvf`. – Mastrianni Jul 17 '20 at 21:36

score 48 · Answer 3 · answered Sep 06 '10 at 13:30

48

If you really don't want to include top directory in the tarball (and that's generally bad idea):

tar czf workspace.tar.gz -C /path/to/workspace .

answered Sep 06 '10 at 13:30

rkhayrov

10,040
2
35
40

I am in need to exclude the top directory and I am in need to place the tar in the base directory. – Micha Sep 06 '10 at 13:54
@Micha make a bash script to archive and copy the file after – Fedir RYKHTIK Jun 07 '13 at 16:19
3

You're missing the - in -czf I think. – UpTheCreek Oct 30 '14 at 09:36
10

man tar: The first argument to tar should be a function; either one of the letters `Acdrtux`, or one of the long function names. **A function letter need not be prefixed with `-`**, and may be combined with other single-letter options. – rkhayrov Oct 30 '14 at 12:17
10

Still doesn't work, I still have the "." as top directory in my tarball – A. Gille Dec 21 '17 at 13:05

score 20 · Answer 4 · answered Oct 02 '14 at 02:13

20

in directory want to compress (current directory) try this :

tar -czf workspace.tar.gz . --exclude=./*.gz

answered Oct 02 '14 at 02:13

empugandring

561
1
5
15

2

But now it complains that the file "." changed while being read. Can I be sure that all the files have been included? – Jaan Jan 04 '15 at 22:41
1

This excludes all `.gz` files in the directory. – omerfarukdogan Jun 05 '18 at 06:45

bramp · Answer 5 · 2020-01-04T17:15:02.867

14

You can include the hidden directories by going back a directory and doing:

cd ..
tar czf workspace.tar.gz workspace

Assuming the directory you wanted to gzip was called workspace.

edited Jan 04 '20 at 17:15

answered Sep 06 '10 at 13:26

bramp

9,581
5
40
46

And this avoids the changing file problem too. – Jonathan Leffler Sep 06 '10 at 15:57
Doesn't work when the user has no write permission in the parent folder. – Joachim Wagner May 24 '17 at 14:45
Exactly @JoachimWagner ! – Adarsh Sojitra Jul 26 '18 at 09:42
1

@JoachimWagner. If you don't have write permission, change it to be: `tar -czf /path/to/where/you/do/have/permission/workspace.tar.gz workspace` – bramp Sep 16 '18 at 20:12
1

I had to take out `-` before the `czf` to make it work. Otherwise I was getting `Cannot stat: No such file or directory` error message. – Phemelo Khetho Oct 29 '19 at 09:03

score 7 · Answer 6 · answered Sep 06 '10 at 14:08

7

You can fix the . form by using --exclude:

tar -czf workspace.tar.gz --exclude=workspace.tar.gz .

answered Sep 06 '10 at 14:08

caf

233,326
40
323
462

score 4 · Answer 7 · answered Aug 24 '14 at 17:12

Actually the problem is with the compression options. The trick is the pipe the tar result to a compressor instead of using the built-in options. Incidentally that can also give you better compression, since you can set extra compresion options.

Minimal tar:

tar --exclude=*.tar* -cf workspace.tar .

Pipe to a compressor of your choice. This example is verbose and uses xz with maximum compression:

tar --exclude=*.tar* -cv . | xz -9v >workspace.tar.xz

Solution was tested on Ubuntu 14.04 and Cygwin on Windows 7. It's a community wiki answer, so feel free to edit if you spot a mistake.

Peter Jaric · Answer 8 · 2010-09-06T14:02:09.290

4

Update: I added a fix for the OP's comment.

tar -czf workspace.tar.gz .

will indeed change the current directory, but why not place the file somewhere else?

tar -czf somewhereelse/workspace.tar.gz .
mv somewhereelse/workspace.tar.gz . # Update

edited Sep 06 '10 at 14:02

answered Sep 06 '10 at 13:43

Peter Jaric

5,162
3
30
42

The file will be processed afterwards and is required to be in the workspace - it is a little bid ugly but it is just needed (the whole thing ist just a workaround ...) – Micha Sep 06 '10 at 13:53
OK, I'll add a fix for that :) – Peter Jaric Sep 06 '10 at 14:00
And if you need to be able to run this repeatedly, add 'rm workspace.tar.gz' before the tar line. – Peter Jaric Sep 06 '10 at 14:03
If you want to tar an entire drive, _into_ that same drive, it seems you don't have many options. – BeeOnRope Jul 30 '17 at 19:32

Joachim Wagner · Answer 9 · 2022-07-04T07:12:45.120

4

Yet another solution, assuming the number of items in the folder is not huge and assuming all names do not contain characters the shell interprets as delimiters (whitespace):

tar -czf workspace.tar.gz `ls -A`

(ls -A prints normal and hidden files but not "." and ".." as ls -a does.)

edited Jul 04 '22 at 07:12

answered May 24 '17 at 14:49

Joachim Wagner

860
7
16

score 2 · Answer 10 · answered Apr 02 '18 at 19:06

Had a similar situation myself. I think it is best to create the tar elsewhere and then use -C to tell tar the base directory for the compressed files. Example:

tar -cjf workspace.tar.gz -C <path_to_workspace> $(ls -A <path_to_workspace>)

This way there is no need to exclude your own tarfile. As noted in other comments, -A will list hidden files.

Ellis · Answer 11 · 2016-03-03T14:31:17.047

The problem with the most solutions provided here is that tar contains ./ at the begging of every entry. So this results in having . directory when opening it through GUI compressor. So what I ended up doing is:

ls -1A | xargs -d "\n" tar cfz my.tar.gz

If you already have my.tar.gz in current directory you may want to grep this out:

ls -1A | grep -v my.tar.gz | xargs -d "\n" tar cfz my.tar.gz

Be aware of that xargs has certain limit (see xargs --show-limits). So this solution would not work if you are trying to create a package which has lots of entries (directories and files) on a directory which you are trying to tar.

score 1 · Answer 12 · answered Sep 06 '10 at 13:34

1

A good question. In ZSH you can use the globbing modifier (D), which stands for "dotfiles". Compare:

ls $HOME/*

and

ls $HOME/*(D)

This correctly excludes the special directory entries . and ... In Bash you can use .* to include the dotfiles explicitly:

ls $HOME/* $HOME/.*

But that includes . and .. as well, so it's not what you were looking for. I'm sure there's some way to make * match dotfiles in bash, too.

answered Sep 06 '10 at 13:34

loevborg

1,774
13
18

In bash, use `.??*` to match all the dot files not `.` and `..`. That doesn't get single character dot files (like `.a`). To get those too, use the more complicated `.[^.]*`. – Scott Minster Jun 10 '13 at 17:20

score 1 · Answer 13 · answered Oct 31 '20 at 20:18

10 years later, you have an alternative to tar, illustrated with Git 2.30 (Q1 2021), which uses "git archive"^(man) to produce the release tarball instead of tar.
(You don't need Git 2.30 to apply that alternative)

See commit 4813277 (11 Oct 2020), and commit 93e7031 (10 Oct 2020) by René Scharfe (rscharfe).
^{(Merged by Junio C Hamano -- gitster -- in commit 63e5273, 27 Oct 2020)}

Makefile: use git init/add/commit/archive for dist-doc

^{Signed-off-by: René Scharfe}

Reduce the dependency on external tools by generating the distribution archives for HTML documentation and manpages using git^(man) commands instead of tar. This gives the archive entries the same meta data as those in the dist archive for binaries.

So instead of:

tar cf ../archive.tar .

You can do using Git only:

git -C workspace init
git -C workspace add .
git -C workspace commit -m workspace
git -C workspace archive --format=tar --prefix=./ HEAD^{tree} > workspace.tar
rm -Rf workspace/.git

That was initially proposed because, as explained here, some exotic platform might have an old tar distribution with lacking options.

bicycle · Answer 14 · 2015-10-08T09:37:15.920

0

If disk space space is not an issue, this could also be a very easy thing to do:

mkdir backup
cp -r ./* backup
tar -zcvf backup.tar.gz ./backup

edited Oct 08 '15 at 09:37

answered Oct 01 '15 at 09:54

bicycle

8,315
9
52
72

score 0 · Answer 15 · answered Sep 06 '10 at 14:15

tar -czf workspace.tar.gz .??* *

Specifying .??* will include "dot" files and directories that have at least 2 characters after the dot. The down side is it will not include files/directories with a single character after the dot, such as .a, if there are any.

Andrew Gunnerson · Answer 16 · 2013-07-16T05:34:33.387

-2

Using find is probably the easiest way:

find . -maxdepth 1 -exec tar zcvf workspace.tar.gz {} \+

find . -maxdepth 1 will find all files/directories/symlinks/etc in the current directory and run the command specified by -exec. The {} in the command means file list goes here and \+ means that the command will be run as:

tar zcvf workspace.tar.gz .file1 .file2 .dir3

instead of

tar zcvf workspace.tar.gz .file1
tar zcvf workspace.tar.gz .file2
tar zcvf workspace.tar.gz .dir3

edited Jul 16 '13 at 05:34

answered May 23 '13 at 00:39

Andrew Gunnerson

638
1
8
17

-J is --xz in the manual; combined with -c is just the start of what a computer will call "fun" :) – Ricalsin Jul 14 '13 at 04:11
Haha, my mini-server at home is more limited by disk space than by the CPU :) Good point though; I'll change it to use gzip. – Andrew Gunnerson Jul 16 '13 at 05:34

tar: add all files and directories in current directory INCLUDING .svn and so on

16 Answers16

`Makefile`: use git init/add/commit/archive for dist-doc

Linked

tar: add all files and directories in current directory INCLUDING .svn and so on

16 Answers16

Makefile: use git init/add/commit/archive for dist-doc

Linked

`Makefile`: use git init/add/commit/archive for dist-doc