1

I'm trying to do some single-cell RNA sequencing analysis. For now, I'm dealing with matrices that represent cells X genes. I'm trying to merge three of these matrices together, and two of them are in the sparse format. I've done this before with matrices with about the same number of columns, but with less rows.

However, when I try to write a csv file out of the merged matrix, I get the following error output:

Error in write.table(test_dataset_m, file = "Splits_Arcuate.csv", sep = ",",  :  
corrupt matrix -- dims not not match length

I'm running this on my Ubuntu 18.04.2 LTS server, using Putty for the SSH connection with my Windows 10 notebook, and I'm running Rstudio on the server as a rocker image (rocker/tidyverse).

I've already tried to eliminate non-expressed genes (colsums = 0), but that doesn't really changes the overall size of the matrix. I've also tried to convert the sparse matrix directly in the write.table command (), but I get an error output.

write.table(as.matrix(test_dataset), file = "Splits_Arcuate.csv", sep = ",", row.names = T, col.names = T)
Error in asMethod(object) : 
  Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 105

For now, here's the code I have. Here I put my rownames together into a list, then use it to find common genes across different datasets and build new matrices with this. I then try to merge the datasets with cbind. split2 and split11 are sparse matrices.

testgenes <- list(split2_genes, split11_genes, rownames(camp_counts))

test_universe <- Reduce(dplyr::intersect, testgenes, accumulate = F)

split2 <- splitseq_dge[test_universe,cells_split2]
split11 <- splitseq_dge[test_universe, cells_split11]
arcuate <- camp_counts[test_universe,]

split2 <- as.matrix(split2)
split11 <- as.matrix(split11)

test_dataset <- cbind(split2, split11, arcuate)

write.table(test_dataset, file = "Splits_Arcuate.csv", sep = ",", row.names = T, col.names = T)

The thing is I need the CSV output, to give as an input to another software.

I'm not sure if this is a memory allocation error. I'm running this on a 16-cores 128GB RAM university server.

Dale K
  • 25,246
  • 15
  • 42
  • 71
Davi Sidarta
  • 11
  • 1
  • 1
    can you try `data.table::fwrite()` to see if it is a memory issue. it is hard to tell without the data. – Mike Apr 08 '19 at 18:24
  • 1
    It's easier to help you if you include a simple [reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) with sample input and desired output that can be used to test and verify possible solutions. Specify any non-base R packages you are using. – MrFlick Apr 08 '19 at 18:25
  • if your matrix is actually sparse format , you can write out in triplet form (`?Matrix::writeMM`) -- but this depends on whether the other software can read this (so you will, at least, need to remove the `as.matrix` calls). Otherwise, [R list](https://stat.ethz.ch/pipermail/r-devel/2018-April/075834.html) suggests writing out in chunks – user20650 Apr 08 '19 at 19:38

0 Answers0