Read and write csv.gz file in R

Question

There are very similar questions about this topic, but non deals with this under R quite precisely.

I have a csv.gz file and I would like to "unzip" the file and have it as ordinary *.csv file. I suppose one would go about first reading the csv.gz file and latter via write.csv command create the csv file itself.

Here what I have tried a part of other things:

gz.file <- read.csv(gzfile(file.choose()), as.is = TRUE)

gives:

  head(gz.file)
        farmNo.milk.energy.vet.cows
  1     1;862533;117894;21186;121
  2     2;605764;72049;43910;80
  3     3;865658;158466;54583;95
  4     4;662331;66783;45469;87
  5     5;1003444;101714;81625;125
  6     6;923512;252408;96807;135

File claims to be data.frame but doesn't behave like one, what I'm missing here?

 class(gz.file)
 [1] "data.frame"

Once read into memory I would like to have it in pure csv file, so would write.csv would be the solution?

write.csv(gz.file, file="PATH")

You want `read.csv2` instead of `read.csv` to deal with semicolon separators. — Thomas, Dec 16 '13 at 11:27
Thanks Tomas. But it seems that just sep = ";" in ordinary read.csv solved my issues. — Maximilian, Dec 16 '13 at 11:32

jangorecki · Answer 1 · 2019-12-10T15:28:55.400

33

In recent versions of data.table fast csv reader fread got support for csv.gz files. It automatically detects if it needs to decompress based on the filename so there is not much new to learn. Following should work.

library(data.table)
dt = fread("data.csv.gz")

This feature requires extra, fortunately lightweight, dependency as you can read in ?fread manual

Compressed files ending .gz and .bz2 are supported if the R.utils package is installed.

To write compressed argument use fwrite(compress="gzip").

edited Dec 10 '19 at 15:28

answered Jan 26 '19 at 14:14

jangorecki

16,384
4
79
160

This is the answer, as far as I'm concerned. I hadn't realized data.table had added that functionality. Just make sure you have R utils `install.packages('R.utils')`. Don't even need to restart your session. – Paul 'Joey' McMurdie Nov 01 '21 at 22:07

liborm · Answer 2 · 2020-10-16T12:17:14.703

11

tidyverse, particularly the readr package, has transparent support of gzip compressed files (and a few others)

library(readr)

read_csv("file.csv.gz") -> d

# write uncompressed data
d %>% write_csv("file.csv")

edited Oct 16 '20 at 12:17

answered Feb 25 '19 at 22:58

liborm

2,634
20
32

Just for completeness sake, the functions `read_csv` and `write_csv` are from the readr package. – Guilherme Marthe Mar 11 '20 at 17:21
In the code it should be library(tidyverse) (you are missing the y en tidyverse) – Matias Jun 06 '20 at 12:29

Read and write csv.gz file in R

2 Answers2

Linked