1

In these days I have run into a problem of data export from R to a more “common” format as .csv or .txt. My dataset is in data.table format and has 149000 rows * 124 columns. I adopt the following lines of code to try to export it:

write.table(data_reduced,"directory/data_reduced.txt",sep="\t",row.names=FALSE)
write.csv2(data_reduced,"directory/data_reduced.csv")

The result, in both cases, is that the .txt or .csv files have a lower number of rows than they are supposed to do and this changes with the different trials I did (it ranges from 900 to 1800, more or less). Usually what I get are the first rows and then the very last one. I have tried to transform the data.table in a matrix or data.frame but the result I get is more or less the same. I have also tried to adopt the write.xlsx function but I have some problems with Java (which is something common as I have noticed reading the SO forum and other web sources).

I have also read about a function called fwrite to export very large datasets but it looks like that my RStudio cannot find it, despite I installed the data.table package.

Can anyone give me an explanation/solution for this problem? I've been reading different sources to sort it out but with no success until now.

I use RStudio Version 0.99.473.

David Arenburg
  • 91,361
  • 17
  • 137
  • 196
Igor Sarman
  • 11
  • 1
  • 3
  • Did you look at the result using a simple text editor or something like Excel? – talat Aug 24 '16 at 12:45
  • 1
    http://stackoverflow.com/questions/12013953/write-csv-for-large-data-table-in-r, for `fwrite` may be you need to update R studio – user5249203 Aug 24 '16 at 13:44
  • Expand on "looks like... my RStudio cannot find it..." -- what is your error?' – MichaelChirico Aug 24 '16 at 18:27
  • @docendodiscimus That's exactely what I did, when I export my data.table in .txt or .csv and I try to open them in a text file or excel what I obtain is a "censored" version of the original dataset. – Igor Sarman Aug 25 '16 at 16:14
  • @MichaelChirico what I obtain is "Error: could not find function "fwrite"" although I have and use the data.table package – Igor Sarman Aug 25 '16 at 16:19
  • post your `sessionInfo` – MichaelChirico Aug 25 '16 at 16:38
  • Possible duplicate of [How to export a large dataset from R to CSV?](http://stackoverflow.com/questions/34217169/how-to-export-a-large-dataset-from-r-to-csv) – polka Aug 25 '16 at 20:07
  • This is a weird problem. Maybe you'll have better luck with the readr package. It's generally better than base r for this stuff. The functions you want are write_csv, write_tsv and write_delim – Max F Nov 11 '21 at 22:39

0 Answers0