Write new file without duplicate with r

Question

I have read a txt file into an R data.frame. Some lines are duplicated. How can i write to a new file without a duplicate (only one of the duplicates for each duplicate) For example:

A; a
B; a
C; a
A; a
C; a
A; b

I must write to new file:

A; a
B; a
C; a
A; b

I tried. My code:

#read file 
t = read.table('/home/BigClaster.txt',sep=';',header = FALSE)
........

I have big file ~ 1269821 lines in txt file. When i read file RStudio in Environment show me line size without duplicate (1,095,079)

When i rewrite to new file i get duplicate lines

Try `df1[!duplicated(df1),]` or `unique(df)` – akrun Apr 23 '18 at 09:02 — akrun, Apr 23 '18 at 09:02

score 1 · Answer 1 · answered Apr 23 '18 at 09:05

1

R base

 t[!duplicated(t), ]

Dplyr

t %>% distinct(.keep_all = TRUE)

Result

  V1 V2
1  A  a
2  B  a
3  C  a
6  A  b

answered Apr 23 '18 at 09:05

Rana Usman

1,031
7
21

when i tryed this code t[!duplicated(t), ] , i get many little lines of my data. My txt file is 1269821 lines. after t[!duplicated(t), ] i get 286,535 lines – Dossanov Apr 23 '18 at 09:44
@Dossan it only returns non duplicated. – Rana Usman Apr 23 '18 at 10:46

Write new file without duplicate with r

1 Answers1