Compare one row to all other rows in a file using R

Question

I have a file like below:

P1 A,B,C    
P2 B,C,D,F    
P3 C,D,E,F

and I need to compare each row to all other rows to get a count of intersecting elements like below:

P1 P2 2    
P1 P3 1    
P2 P3 3

Thank you,
S

It's going to depend on how you are representing the P's, although as lists of characters would seem to be the only sensible option. — IRTFM, Sep 02 '11 at 22:20

score 4 · Answer 1 · answered Sep 02 '11 at 22:20

It's unclear where the original data is coming from, so I assumed that you read the data into a data.frame as below:

x <- data.frame(V1 = c("a", "b", "c"), 
                V2 = c("b", "c", "d"), 
                V3 = c("c", "d", "e"),
                V4 = c(NA, "f", "f"),
                stringsAsFactors = FALSE
                )

row.names(x) <- c("p1", "p2", "p3")

The first step is to create the combination of all rows that you need to compare:

rowIndices <- t(combn(nrow(x), 2))
> rowIndices
     [,1] [,2]
[1,]    1    2
[2,]    1    3
[3,]    2    3

Then we can use that information in apply with length() and intersect() to get what you want. Note I also indexed into the row.names() attribute of the data.frame x to get the row names like you wanted.

data.frame(row1 = row.names(x)[rowIndices[, 1]], 
      row2 = row.names(x)[rowIndices[, 2]],
      overlap = apply(rowIndices, 1, function(y) length(intersect(x[y[1] ,], x[y[2] ,])))
      )

Gives you something like:

  row1 row2 overlap
1   p1   p2       2
2   p1   p3       1
3   p2   p3       3

score 2 · Accepted Answer · answered Sep 03 '11 at 14:26

Read example data.

txt <- "P1 A,B,C
        P2 B,C,D,F
        P3 C,D,E,F"
tc <- textConnection(txt)
dat <- read.table(tc,as.is=TRUE)
close(tc)

Transform to long format and use self join with aggregating function.

dat_split <- strsplit(dat$V2,",")
dat_long <- do.call(rbind,lapply(seq_along(dat_split),
            function(x) data.frame(id=x,x=dat_split[[x]], stringsAsFactors=FALSE)))

result <- sqldf("SELECT t1.id AS id1,t2.id AS id2,count(t1.x) AS N 
                 FROM dat_long AS t1  INNER JOIN dat_long AS t2 
                 WHERE (t2.id>t1.id) AND (t1.x=t2.x) GROUP BY t1.id,t2.id")

Results

> result
  id1 id2 N
1   1   2 2
2   1   3 1
3   2   3 3

This is a scalable solution because it works with large files. Thank you — user602599, Sep 04 '11 at 02:53

Compare one row to all other rows in a file using R

2 Answers2

Linked