How to get some subset of data from a csv file for big-data(comparing csv's)?

Question

I had 2 csv files, first csv file has 16749 rows,with SYMBOLS and log as a columns. In the first csv it contains some junk data, which is not needed at all. Second, is a clean csv file (no LOC11*) has 14107 rows, with SYMBOLS as only one column. I'd like to add for each SYMBOL ID it should find the corresponding logFC value from the first csv file. I'm breaking my head, newbie to programming, unable to do it. Some help please.

Example:
First csv
SYMBOL          log
LOC117745510   -0.35
LOC117741824   -0.54
ipmkb          -0.46
prrc2b

Second csv (log column to be added from 1st csv as reference)
SYMBOL          log
hs6st1a
ipmkb
prrc2b

There is a command line utility called `join` in GNU coreutils. Just do on command line: `join <(sort first.csv) <(sort second.csv)` — M. Nejat Aydin, Jul 03 '20 at 22:08
I did already: `join <(sort first.csv) <(sort second.csv)`. first.csv and second.csv are filenames which contain the data as you specified in the question. — M. Nejat Aydin, Jul 04 '20 at 00:38
Yes,I tried it not working. Tables looks same, I tried to write the output to the new file, it empty. — park, Jul 04 '20 at 01:05

eastclintw00d · Accepted Answer · 2020-07-04T19:57:59.693

0

Read in your data with an appropriate import function like read.csv and then join it.

library(dplyr)

## read your files (possibly you need to adjust some arguments in read.csv)
file1 <- read.csv("path/to/file1.csv", header = TRUE)
file2 <- read.csv("path/to/file2.csv", header = TRUE)

file2 %>%
  left_join(file1, by = "SYMBOL)

edited Jul 04 '20 at 19:57

answered Jul 03 '20 at 21:28

eastclintw00d

2,250
1
9
18

How to get some subset of data from a csv file for big-data(comparing csv's)?

1 Answers1