2

I'm trying to delete matching patterns, starting from the second occurrence, using sed or awk. The input file contains the information below:

abc
def
abc
ghi
jkl
abc
xyz
abc

I want to the delete the pattern abc from the second instance. The output should be as below:

abc
def
ghi
jkl
xyz
Rubens
  • 14,478
  • 11
  • 63
  • 92
Krish
  • 23
  • 1
  • 3

4 Answers4

6

Neat sed solution:

sed '/abc/{2,$d}' test.txt
abc
def
ghi
jkl
xyz
Rubens
  • 14,478
  • 11
  • 63
  • 92
  • This won't work if the first `abc` pattern were to happen on 2nd line or later. This command just ignores the first line.. parses from the 2nd line till the last: `2,$`. – Kaushal Modi Jan 27 '15 at 14:37
3
$ awk '$0=="abc"{c[$0]++} c[$0]<2; ' file
abc
def
ghi
jkl
xyz

Just change the "2" to "3" or whatever number you want to keep the first N occurrences instead of just the first 1.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
2

One way using awk: 

$ awk 'f&&$0==p{next}$0==p{f=1}1' p="abc" file
abc
def
ghi
jkl
xyz

Just set p to pattern that you only want the first instance of printing:

Chris Seymour
  • 83,387
  • 30
  • 160
  • 202
1

Taken from : unix.com

Using awk '!x[$0]++' will remove duplicate lines. x is a array and it's initialized to 0.the index of x is $0,if $0 is first time meet,then plus 1 to the value of x[$0],x[$0] now is 1.As ++ here is "suffix ++",0 is returned and then be added.So !x[$0] is true,the $0 is printed by default.if $0 appears more than once,! x[$0] will be false so won't print $0.

Binayaka Chakraborty
  • 1,325
  • 9
  • 14
  • This will also remove any duplicates of any other patterns. I think the OP just want the duplicates of a given pattern removing. – Chris Seymour Jul 28 '13 at 17:24