489

I want a bash command that I can pipe into that will sum a column of numbers. I just want a quick one liner that will do something essentially like this:

cat FileWithColumnOfNumbers.txt | sum
Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
Jubal
  • 8,357
  • 5
  • 29
  • 30
  • Related: http://stackoverflow.com/q/3934423/376454 – Wok Nov 19 '13 at 16:02
  • FYI the `awk` solution is both [easier to remember](https://stackoverflow.com/questions/450799/shell-command-to-sum-integers-one-per-line#comment12469220_451204) and about 2x faster (see [here](https://stackoverflow.com/a/3098072/52074) and [here](https://stackoverflow.com/questions/450799/shell-command-to-sum-integers-one-per-line#comment20238060_451204)). – Trevor Boyd Smith Apr 10 '19 at 14:03

10 Answers10

1081

Using existing file:

paste -sd+ infile | bc

Using stdin:

<cmd> | paste -sd+ | bc

Edit: With some paste implementations you need to be more explicit when reading from stdin:

<cmd> | paste -sd+ - | bc

Options used:

-s (serial) - merges all the lines into a single line

-d - use a non-default delimiter (the character + in this case)

Dimitre Radoulov
  • 27,252
  • 4
  • 40
  • 48
  • 77
    There should be a badge for this. – Amardeep AC9MF Jun 23 '10 at 00:17
  • 13
    Just as an FYI, the `-s` option is in GNU `paste`; it is not supported by Mac OS X 10.7.4 `paste`. However, since the POSIX 2008 specification of [`paste`](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/paste.html) supports `-s`, this is a deficiency in the Mac OS X version. – Jonathan Leffler Aug 15 '12 at 18:56
  • 23
    Just another FYI for OS X, I had to add a - at the end of the paste command in order for this to work on OS X 10.6.8. – Clayton Stanley Nov 09 '12 at 16:48
  • 8
    With `bc -l` you can even add up float numbers. With `cut` you can select columns from the input: `cat input.txt | cut -d ' ' -f 8 | paste -sd+ | bc -l` will add all float numbers in column 8 of the input (space being the field separator). – Arne Sep 24 '13 at 14:59
  • 32
    Well, that would be easier with awk: `awk 'END { print s } { s += $8 }' infile` :) – Dimitre Radoulov Sep 24 '13 at 15:31
  • 1
    bc doesn't seem to support scientific notation. I have decimal numbers some of which are like 2.345e-3. Thoughts? Perl/Python? – Ben May 13 '14 at 19:48
  • 2
    @Ben, in the thread below you'll find _awk_ and _Perl_ solutions. – Dimitre Radoulov May 14 '14 at 07:31
  • 3
    Sorry, all you paste lovers. -1 from me for being too specific. This works ONLY if you have nothing but numbers in your file. I almost have never had that situation. Google "bash add column numbers" and you get here. Invariably, I am trying to add a column interspersed with other data (du output or the like- we are talking bash, the sysadmin's language, right?). Ghostdog's awk answer works with any data that includes a column of numbers AND other columns. It's much less fragile, almost as concise, and faster to boot. – Mike S May 06 '16 at 15:13
  • Had to add float numbers `bc -l` gave parse error. Using `octave` instead of `bc` worked nicely. – Axel Bregnsbo Nov 01 '16 at 12:19
  • @AxelBregnsbo, it must be a decimal separator issue (, vs. .), because obviously bc supports floating point calculations. – Dimitre Radoulov Nov 02 '16 at 09:02
  • paste and bc don't come out of the box, while awk does, so I prefer awk. – mariotomo Feb 08 '18 at 10:43
  • 2
    ` | paste -sd+ - | bc ` was like a magic to me. bash command line rocks! – A. K. Nov 08 '18 at 00:06
  • This only works if there is exactly one column in the file. – gerrit Dec 21 '18 at 12:42
  • 1
    I didn't know about the paste command. Nice trick :) – rkachach Mar 14 '19 at 16:47
  • 2
    this has lots of upvotes but can the answer include what all the commands and switches are actually doing? – red888 Mar 10 '20 at 21:07
  • I agree with @red888's comment; the answer could really benefit from that. Additionally, I see the `d` option being used in `-sd+` but I don't see a character being given as a delimeter? How does that work? – Guy Apr 28 '21 at 05:34
  • 1
    On mac os I had to do: `command | paste -s -d'+' - | bc` instead – Samuel Prevost Jul 25 '21 at 17:20
237

I like the chosen answer. However, it tends to be slower than awk since 2 tools are needed to do the job.

$ wc -l file
49999998 file

$ time paste -sd+ file | bc
1448700364

real    1m36.960s
user    1m24.515s
sys     0m1.772s

$ time awk '{s+=$1}END{print s}' file
1448700364

real    0m45.476s
user    0m40.756s
sys     0m0.287s
ghostdog74
  • 327,991
  • 56
  • 259
  • 343
  • 3
    Good point! On SunOS 5.8 bc even core dumps with such a big input file (see my post below). – Dimitre Radoulov Jun 23 '10 at 08:10
  • 4
    awk is the correct tool for the job! The bc solution is OK but what happens when you need to sum two columns or perhaps filter out negative numbers. With awk you can easily and sensibly add in extra logic, with the bc solution you end up with piping through yet another command (cut or grep) – James Anderson Jun 23 '10 at 08:26
  • @radoulov. thanks. didn't know Solaris's bc balks on big inputs. – ghostdog74 Jun 23 '10 at 12:43
  • 2
    I would use awk also because it allows to initialize `s` to 0, by doing `awk 'BEGIN{s=0}{s+=$1}END{print s}'`. In this case, if an empty column is piped into `paste | bc`, the latter pipe will return `0` instead of null. (There may be cases where and empty column is a legitimate input). – gbgnv Mar 31 '15 at 22:12
  • This is much more general solution, faster, and less fragile than the chosen answer. +1 as the best answer to "How to add a column of numbers in a file." The chosen answer covers "How to add ONLY a column of numbers in a file." – Mike S May 06 '16 at 15:14
  • This solution seems to round down though to a single decimal? I have a list of numbers which happen to be to 2 decimal places (dollar/cents). With the paste/bc I get 11027.24, but with the awk solution I get 11027.2. That's a pretty critical bug if you end up using this one-liner with monetary values! – romemmy Jun 13 '19 at 22:47
  • 1
    @romemmy Never fear, awk can do precision: `awk '{s+=$1}END{printf "%.2f\n",s}' /tmp/py.venv.strace` . I'll update the answer. – Mike S Nov 18 '22 at 16:05
  • `awk` handles empty lines `paste -sd+ -` do not – Kangur Jul 26 '23 at 12:36
96

The following command will add all the lines(first field of the awk output)

awk '{s+=$1} END {print s}' filename
codeforester
  • 39,467
  • 16
  • 112
  • 140
minhas23
  • 9,291
  • 3
  • 58
  • 40
74

Does two lines count?

awk '{ sum += $1; }
     END { print sum; }' "$@"

You can then use it without the superfluous 'cat':

sum < FileWithColumnOfNumbers.txt
sum   FileWithColumnOfNumbers.txt

FWIW: on MacOS X, you can do it with a one-liner:

awk '{ sum += $1; } END { print sum; }' "$@"
Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
  • @jskaggz - see my answer for a bit shorter/simpler Perl version :) – DVK Jun 22 '10 at 19:11
  • 9
    the one-liner awk works on any awk that I've tried, not just OS X – unhammer May 24 '13 at 06:57
  • @unhammer: I don't have access to 7th Edition UNIX and its version of `awk` any more. I don't know whether the one-line would have worked there, but I didn't try it back then when I learned `awk`. You're probably right; all current versions of `awk` are likely to support the one-liner. – Jonathan Leffler May 24 '13 at 06:59
  • what does the $@ do? It seems to have no effect for my usage of pbpaste | awk ... – David Mann Aug 10 '16 at 14:49
  • @DavidMann: `"$@"` is used inside a script to represent all the arguments to the script, or nothing/none if there were no arguments. If you're using the `awk` in `pbpaste | awk …`, you simply omit the `"$@"` (though it would usually do no damage; most interactive shells at a terminal have no 'positional parameters' so `"$@"` is nothing). But if you have a shell script `sumcol1.sh`, adding the `"$@"` would be sensible — you'd then use `pbpaste | sumcol1.sh` or similar and it would work correctly, and so would `sumcol1.sh file1 file2`. – Jonathan Leffler Aug 10 '16 at 14:51
19

[a followup to ghostdog74s comments]

bash-2.03$ uname -sr
SunOS 5.8

bash-2.03$ perl -le 'print for 1..49999998' > infile

bash-2.03$ wc -l infile
 49999998 infile

bash-2.03$  time paste -sd+ infile | bc
bundling space exceeded on line 1, teletype
Broken Pipe

real    0m0.062s
user    0m0.010s
sys     0m0.010s

bash-2.03$ time nawk '{s+=$1}END{print s}' infile
1249999925000001

real    2m0.042s
user    1m59.220s
sys     0m0.590s
bash-2.03$ time /usr/xpg4/bin/awk '{s+=$1}END{print s}' infile
1249999925000001

real    2m27.260s
user    2m26.230s
sys     0m0.660s

bash-2.03$ time perl -nle'
  $s += $_; END { print $s }
   ' infile
1.249999925e+15

real    1m34.663s
user    1m33.710s
sys     0m0.650s
Dimitre Radoulov
  • 27,252
  • 4
  • 40
  • 48
  • 2
    +1 for pointing out the coredump in the bc version of this. I'm hesitant to change the "answer" of this question because the bc version works fine for me (i'm totalling up 30 numbers). – Jubal Jun 23 '10 at 16:58
  • +1 for the perl version - fastest on my ubuntu 12.04 – drevicko Jun 27 '13 at 05:36
  • +1 for Perl version working on floating-point numbers. Also, for exactly what I was looking for, here's summing a particular column (column 2 indexed from zero): `perl -nle '$s += (split)[2]; END { print $s }' foo.txt` or using pipes: `cat foo.txt | perl -nle '$s += (split)[2]; END { print $s }'`. – Ben May 14 '14 at 13:17
  • Also: `paste -sd+ -|perl -nle 'print eval'` – Ben May 14 '14 at 13:29
15

You can use bc (calculator). Assuming your file with #s is called "n":

$ cat n
1
2
3
$ (cat n | tr "\012" "+" ; echo "0") | bc 
6

The tr changes all newlines to "+"; then we append 0 after the last plus, then we pipe the expression (1+2+3+0) to the calculator

Or, if you are OK with using awk or perl, here's a Perl one-liner:

$perl -nle '$sum += $_ } END { print $sum' n
6
DVK
  • 126,886
  • 32
  • 213
  • 327
11
while read -r num; do ((sum += num)); done < inputfile; echo $sum
Dennis Williamson
  • 346,391
  • 90
  • 374
  • 439
  • 1
    If you've a lot of numbers to add, this ain't so good: using @radulov's file, this took 10m28s (Ubuntu 12.04). The perl solution took 11s, awk 13s and |bc 37s. – drevicko Jun 27 '13 at 05:35
  • 1
    @drevicko On the other hand, if there is not a lot of numbers to add (jskaggz tell his need is for 30 numbers), this way is the quickest, because there is no fork! – techno Oct 10 '13 at 07:27
  • @techno True enough, though there's only one fork. On the other hand, for 30 numbers, speed is a bit irrelevant - best to go with the tool you're most familiar with, such as bash (: – drevicko Oct 15 '13 at 04:48
  • He-he, this is in fact the first answer to question asked ("how to sum column with **bash**"). – Alex Offshore Jan 21 '20 at 08:47
5

Use a for loop to iterate over your file …

sum=0; for x in `cat <your-file>`; do let sum+=x; done; echo $sum
t6d
  • 2,595
  • 3
  • 26
  • 42
4

If you have ruby installed

cat FileWithColumnOfNumbers.txt | xargs ruby -e "puts ARGV.map(&:to_i).inject(&:+)"
Joe Cannatti
  • 4,989
  • 5
  • 38
  • 56
-7
[root@pentest3r ~]# (find / -xdev -size +1024M) | (while read a ; do aa=$(du -sh $a | cut -d "." -f1 ); o=$(( $o+$aa )); done; echo "$o";)
khagler
  • 3,996
  • 29
  • 40
paco
  • 1
  • 6
    There's part of an answer to question buried in some rather obscure script that finds big files and runs `du` on each file separately, which is both painful and largely irrelevant. Another, older answer provides the same 'sum it using just shell' solution. Given the absence of explanation and repetition of another answer and off-topic coding, there really isn't any benefit to keeping this answer. – Jonathan Leffler Jul 30 '15 at 19:50