combine like terms in bash

Question

I have a list of domain names in a text file with a number of times they occur in a collection of email files. For example:

 598 aol.com
  1 aOL.COM
  4 Aol.com
  1 AOl.com
  6 AOL.com
 39 AOL.COM

There were 598 emails sent to aol.com and 1 sent to aOL.COM and so on. I was wondering if there was a way in bash to combine aol.com and aOL.COM and all the other aliases since they are in fact the same thing. Any help would be greatly appreciated!

This is the line of code that produced that output:

grep -E -o -r "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b" $ARCHIVE | sed 's/.*@//' | sort | uniq -c > temp2

Robby Cornelissen · Answer 1 · 2015-04-21T03:40:33.880

2

Add a -i (--ignore-case) flag to the uniq command in your one-liner:

grep -E -o -r "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b" $ARCHIVE \
    | sed 's/.*@//' \
    | sort \
    | uniq -ic > temp2

From the uniq man page:

-i
--ignore-case
    Ignore differences in case when comparing lines.

edited Apr 21 '15 at 03:40

answered Apr 21 '15 at 03:37

Robby Cornelissen

91,784
22
134
156

1

@Albert, This is actually easier, I would recommend this, but understand why it works. – Tyler Scott Apr 21 '15 at 03:39

score 0 · Answer 2 · edited May 23 '17 at 10:10

0

I would recommend changing the program producing this code to first make everything lowercase, (Converting string to lower case in Bash shell scripting), then try sorting.

Doing this after the fact would just make your life harder.

edited May 23 '17 at 10:10

Community

1
1

answered Apr 21 '15 at 03:34

Tyler Scott

1,046
17
33

combine like terms in bash

2 Answers2

Linked