Remove duplicate lines with 1 command

Asked Mar 09 '20 at 16:54

Active Mar 09 '20 at 17:17

Viewed 568 times

There is any Linux command that deletes duplicates of lines in a file but not the first appearance of the line that is duplicate.

For example, we have a file that has the following lines:

Test Line
Another line
Test
4th line
Test
Test Line

And I want after I run the command, the lines that should remain in the file are:

Test Line
Another Line
Test
4th line

edited Mar 09 '20 at 16:56

P....

asked Mar 09 '20 at 16:54

Robert Gutu

did you try `awk '!a[$0]++' inputfile` ? – P.... Mar 09 '20 at 16:57
this command just show the file without the duplicates , but it doesnt really delete the duplicate lines in the original file – Robert Gutu Mar 09 '20 at 17:22
search for in-place replacement using `awk '{...}' input >input.tmp && mv input.tmp input` construct – P.... Mar 09 '20 at 17:37

0 Answers0