Calculate size of files in shell

Question

I'm trying to calculate the total size in bytes of all files (in a directory tree) matching a filename pattern just using the shell. This is what I have so far:

find -name *.undo -exec stat -c%s {} \; | awk '{total += $1} END {print total}'

Is there an easier way to do this? I feel like there should be a simple du or find switch that does this for me but I can't find one.

To be clear I want to total files matching a pattern anywhere under a directory tree which means

du -bs *.undo

won't work because it only matches the files in the current directory.

Anyway, `du` (or `ls`) won't work above a few thousands of files (" Argument list too long" error). — Skippy le Grand Gourou, Aug 14 '13 at 09:36

score 97 · Accepted Answer · edited Jan 02 '12 at 16:25

97

Try:

find . -name "*.undo" -ls | awk '{total += $7} END {print total}'

On my system the size of the file is the seventh field in the find -ls output. If your find … -ls output is different, adjust.

In this version, using the existing directory information (file size) and the built-in ls feature of find should be efficient, avoiding process creations or file i/o.

edited Jan 02 '12 at 16:25

Gilles 'SO- stop being evil'

104,111
38
209
254

answered Mar 01 '09 at 01:59

tpgould

1,746
10
10

2

That should work, but the -ls option to find, and awk, are overkill for this task. – David Z Mar 01 '09 at 07:55
4

I would add "-type f" to the find command to prevent from incorrect total if there are directories matching "*.undo" glob. – Michał Šrajer Apr 15 '11 at 08:14
2

Note that if you need several patterns to match, you will have to use escaped parenthesis for the whole expression to match otherwise the `-ls` will apply only to the last pattern. For instance, if you want to match all jpeg and png files (trusting filenames), you would use `find . $ -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.png" $ -ls | awk '{total += $7} END {print total}'` (`-iname` is for case insensitive search ; also, note the space between the expression and the escaped parenthesis). – Skippy le Grand Gourou Aug 14 '13 at 09:55
1

Is this _really_ the easiest way to do `dir *.undo /s` in Linux? – Luke Puplett Mar 13 '14 at 15:12

score 36 · Answer 2 · answered Mar 01 '09 at 03:00

36

With zsh, you can use extended globbing to do:

du -c **/*.undo

answered Mar 01 '09 at 03:00

Chris AtLee

7,798
3
28
27

score 22 · Answer 3 · answered May 18 '09 at 00:43

22

find -name *.undo -print0 | du -hc --files0-from=-

answered May 18 '09 at 00:43

gerdemb

11,275
17
65
73

This lists all files. Is there a way to show just the total? Regardless, I'm thinking this is a 'correct answer' candidate. – Tyler Collier Jan 10 '14 at 06:11
3

I like this because it shows the size 'human readable'. @Tyler Collier to show only total use tail: `find -type f -name '*.undo' -print0 | du -hc --files0-from=- | tail -n 1` – Florian F Jan 21 '14 at 11:56
This is the best answer I have ever seen for doing this on linux... its also executes about a 1,000 times faster than any other answer I have seen, since du on a file executes so fast, plus it gives output in human readable terms – Myforwik Feb 06 '15 at 00:06
`-files0-from` doesn't work if you have too many files, I got an error about too long file name from it – szx Feb 20 '18 at 11:06

Cantillon · Answer 4 · 2009-03-01T01:37:14.007

18

du -c *pattern*

This will print the total on the last line of output.

edited Mar 01 '09 at 01:37

answered Mar 01 '09 at 01:10

Cantillon

1,588
1
12
23

1

And use tail to trim off all but the last line. +1. =] – strager Mar 01 '09 at 01:29
6

That'll only find the files that match the pattern in the current directory, though - won't it? – Jonathan Leffler Mar 01 '09 at 03:14
You can use a pattern like `*/*.undo`... although that would only find files with the undo extension that are one folder deep... also wouldn't find any in the current directory. – Kasapo Aug 05 '14 at 14:51
Simplest solution so far ! – maxime1992 Jun 05 '15 at 03:29
`du -ch *pattern*` to get the results in human readable format (e.g., 99K 42M 67.8G) – Will Oct 23 '15 at 00:07

score 9 · Answer 5 · edited May 23 '17 at 12:26

I have been looking at this problem too (only a year later...) - only just found this page.

Something that I found works (for me) is the following:

find /mnt/iso -name *.avi -printf "%s\n" | paste -sd+ - | bc

This will return the total size of all the .avi files in all the sub-folders below /mnt/iso

I have to give credit to radoulov for the paste command - see this page: Shell command to sum integers, one per line?

Just to add - just in case a folder matches the search term - it's a good idea to use -type f in the find command too.

David Z · Answer 6 · 2009-03-01T07:54:02.743

8

find -name '*.undo' -exec wc -c {} + | tail -n 1

should give the actual total number of bytes in the files, if you don't have too many files (where "too many" is going to be a really large number, probably in the thousands). Or if you just want to get the number alone,

find -name '*.undo' -exec wc -c {} + | tail -n 1 | cut -d' ' -f 1

edited Mar 01 '09 at 07:54

answered Mar 01 '09 at 01:16

David Z

128,184
27
255
279

You should quote '*.undo', "*.undo", \*.undo, or similar; otherwise, you'll get wrong results if there exists a.undo file in the current directory. – ephemient Mar 01 '09 at 03:48

score 5 · Answer 7 · answered Mar 01 '09 at 01:29

Python is part of most linux distributions.

import os
import fnmatch
size= 0
for path, dirs, files in os.walk( '.' ):
    for f in files:
        if fnmatch.fnmatch(f,'*.py'):
            fileSize= os.path.getsize( os.path.join(path,f) ) 
            print f, fileSize
            size += fileSize
print size

Longish, but perfectly clear and highly extensible.

score 3 · Answer 8 · edited May 23 '17 at 12:34

3

find -name '*.undo' -print0 | du -hc --files0-from=- | tail -n 1

Put together from gerdemb's and strager's contributions. Using du -cb should display bytes.

edited May 23 '17 at 12:34

Community

1
1

answered Apr 13 '10 at 11:14

xilef

2,199
22
16

score 2 · Answer 9 · answered Jul 18 '13 at 18:43

2

I use du command like this to get the number only:

du file_list | awk '{s+=$1} END {print s}'

answered Jul 18 '13 at 18:43

Hoang NT

43
1
6

score 0 · Answer 10 · answered Feb 08 '13 at 18:50

0

How about this simple one.

find ./ -name *.undo | xargs wc

answered Feb 08 '13 at 18:50

Dennis Mathews

6,897
2
26
39

score 0 · Answer 11 · answered Feb 03 '15 at 19:59

Or, you can just do this:

dir=$1

for file in $dir/* ; do

 length_file=`stat -c %s $file`
 echo "File $file has length $length_file"
 length_total=`expr $length_total + $length_file`

done

echo "Total length: $length_total ."

Where stat displays a file or file system status. The argument -c means using the specified format instead of the default one, and the format sequence $s allows the display of the total size of bytes.

expr

Just evaluates an expression.

Chris Koknat · Answer 12 · 2015-10-08T14:50:25.270

0

Perl one-liner:

find . -name "*.undo" -ls | perl -lane '$t += $F[6]; END{print $t}'

The @F autosplit array starts at index $F[0] while awk fields start with $1, hence $F[6] is used instead of awk's $7

edited Oct 08 '15 at 14:50

answered Sep 09 '15 at 17:26

Chris Koknat

3,305
2
29
30

score 0 · Answer 13 · answered Jun 14 '16 at 20:52

0

du -c | awk '/./{line=$0} END{print $1}'

answered Jun 14 '16 at 20:52

Peter Cotton

1,671
14
17

score 0 · Answer 14 · answered May 11 '17 at 16:07

0

I think the version with xargs could be imroved (simplified) ls -1 *.undo | xargs wc

answered May 11 '17 at 16:07

Jacob

1

score -3 · Answer 15 · answered Mar 01 '09 at 01:04

-3

Check the du (disk usage) command.

answered Mar 01 '09 at 01:04

Christian C. Salvadó

807,428
183
922
838

Calculate size of files in shell

15 Answers15

Linked