Stream in CSV content in R

Question

I understand how to read a CSV file that is stored on disk, but I don't know how to stream in CSV content via CLI using R.

E.g., Reading CSV file from disk using a simple CLI.

library(optparse)

option_list <- list(
    # Absolute filepath to CSV file.
    make_option(c("-c","--csv"),type="character",default=NULL,
                help="CSV filepath",metavar="character")
);
opt_parser <- OptionParser(option_list=option_list)
opt <- parse_args(opt_parser)

csv_filepath <- opt$csv
csv <- read.csv(csv_filepath)

How would I do this if I'm working with a data stream?

In case by "stream" you mean "read from url", [this](https://stackoverflow.com/questions/6299220/access-a-url-and-read-data-with-r) might help. — Andre Wildberg, Nov 21 '21 at 16:19
I do not understand your example, because you initialize a variable option_list, which is not used. Then you use a variable opt, which is not defined. Also, what is a stream in this context? Do you want to read from a content which is already in memory? — user2332849, Nov 21 '21 at 16:31
@user2332849 you are right - I've updated the example. The stream is a stream of CSV byte chunks. So yes, read content that is already in memory. In a strange way, I am interfacing the R script via another language and want to 'hot-potato' the CSV stream to R without saving to disk. — irahorecka, Nov 21 '21 at 16:40

score 1 · Answer 1 · answered Nov 21 '21 at 17:22

R always reads from connections. A connection can be a file, and url, an in-memory text, and so on.

So, in case you wanna read csv-format data from a content that is already in memory, you just use the text= parameter, instead of a file name.

Like this:

my_stream = "name;age\nJulie;25\nJohn;26"
read.csv(text = my_stream, sep = ";", header = T)

The output will be:

   name age
1 Julie  25
2  John  26

You can place additional parameters to read.csv() normally, of course.

score 1 · Answer 2 · answered Nov 21 '21 at 17:47

R source and package `optparse`.

First, write an R source file "example.R", such as the following.

#!/usr/bin/env Rscript
#
# R source: example.R
# options:  -c --csv
# 
library(optparse)

option_list <- list(
    # Absolute filepath to CSV file.
    make_option(c("-c","--csv"),type="character",default=NULL,
                help="CSV filepath",metavar="character")
)
opt_parser <- OptionParser(option_list=option_list)
opt <- parse_args(opt_parser)

csv_filepath <- opt$csv
csv <- read.csv(csv_filepath)

message(paste("\nfile read:", csv_filepath, "\n"))
str(csv)

Then, change the execute permissions, in order for the bash shell to recognize the #! shebang and run Rscript passing it the file.
In this case, I will change the user permissions only, not its group.

bash$ chmod u+x example.R

The test.

I have tested the above script with this data.frame:

df1 <- data.frame(id=1:5, name=letters[1:5])
write.csv(df1, "test.csv", row.names=FALSE)

And, at a Ubuntu 20.04 LTS, ran the command ./example.R passing it the CSV filename in argument csv. The command and its output were

bash$ ./example.R --csv=test.csv

file read: test.csv 

'data.frame':   5 obs. of  2 variables:
 $ id  : int  1 2 3 4 5
 $ name: chr  "a" "b" "c" "d" ...

Stream in CSV content in R

2 Answers2

R source and package optparse.

The test.

R source and package `optparse`.