Removing Only Sequential Duplicates from Text FIle?

Asked Oct 08 '21 at 11:07

Active Oct 09 '21 at 20:48

Viewed 24 times

I've got no problem just blanket removing duplicates - sort -u input.txt > output.txt does that easily. But I want to only remove sequential, subsequent duplicates and I'm not sure how to do that. Here's an example:

Input:

One
Two
Two
Three
Four
Five
Five
Six
Seven
Seven
Seven
Three
Three
Two
One

Output:

One
Two
Three
Four
Five
Six
Seven
Three
Two
One

Has anyone worked out a simple bash sequence to do this, or know of a *NIX command line utility that does?

Edit: clarified the example.

edited Oct 09 '21 at 20:48

asked Oct 08 '21 at 11:07

Christopher Penn

`awk '!seen[$0]++' file.txt` – Jetchisel Oct 08 '21 at 11:09
2

Like ... `uniq input.txt`? – Arkadiusz Drabczyk Oct 08 '21 at 11:10
@Christopher What does "sequential, subsequent duplicate" mean? From `One One Two Three One` you want `One Two Three` or `One Two Three One`? – Casaper Oct 08 '21 at 20:08
Thanks for the comments - I clarified the example to show the intended output. Casaper - the latter. – Christopher Penn Oct 09 '21 at 20:49

Removing Only Sequential Duplicates from Text FIle?

0 Answers0