Stack Overflow
Stack Overflow

Questions tagged [fread]

A binary-safe file read function in C/C++/PHP that returns the specified number of bytes from a stream. Also a fast csv parser in R's data.table package.

References:

1613 questions
79
votes
7 answers

How does fread really work?

The declaration of fread is as following: size_t fread(void *ptr, size_t size, size_t nmemb, FILE *stream); The question is: Is there a difference in reading performance of two such calls to fread: char a[1000]; fread(a, 1, 1000, stdin); fread(a,…
Roman Byshko
  • 8,591
  • 7
  • 35
  • 57
42
votes
2 answers

Fast reading and combining several files using data.table (with fread)

I have several different txt files with the same structure. Now I want to read them into R using fread, and then union them into a bigger dataset. ## First put all file names into a list library(data.table) all.files <- list.files(path =…
Bigchao
  • 1,746
  • 3
  • 15
  • 31
39
votes
6 answers

'Embedded nul in string' error when importing csv with fread

I have a large file (3.5G) that I'm trying to import using data.table::fread. It was originally created from an rpt file that was opened as text and saved as a CSV. This has worked fine with smaller files (of the same type of data-same columns and…
datahappy
  • 826
  • 2
  • 11
  • 29
38
votes
5 answers

How do you read multiple .txt files into R?

I'm using R to visualize some data all of which is in .txt format. There are a few hundred files in a directory and I want to load it all into one table, in one shot. Any help? EDIT: Listing the files is not a problem. But I am having trouble going…
Eric Brotto
  • 53,471
  • 32
  • 129
  • 174
37
votes
5 answers

fgets() and fread() - What is the difference?

I understand the differences between fgets() and fgetss() but I don't get the difference between fgets() and fread(), can someone please clarify this subject? Which one is faster? Thanks!
Alix Axel
  • 151,645
  • 95
  • 393
  • 500
36
votes
1 answer

using colClasses in fread

I don't know how to choose specific columns using the colClasses option in fread. I tried to use NULL in several ways but nothing worked. Here's a minimal example. I just want columns 1 and 3. dt <-…
beginneR
  • 3,207
  • 5
  • 30
  • 52
32
votes
4 answers

Reading in chunks at a time using fread in package data.table

I'm trying to input a large tab-delimited file (around 2GB) using the fread function in package data.table. However, because it's so large, it doesn't fit completely in memory. I tried to input it in chunks by using the skip and nrow arguments such…
FBC
  • 993
  • 3
  • 9
  • 15
30
votes
5 answers

PHP technique to query the APNs Feedback Server

Can someone clarify what the APNs (Apple Push Notification) wants as far as how you query it? The docs say it starts sending as soon as the connection is made. Does this mean that I don't do an fread() on it? Here's my current code to try and read…
Johnny
28
votes
1 answer

Reason behind speed of fread in data.table package in R

I am amazed by the speed of the fread function in data.table on large data files but how does it manages to read so fast? What are the basic implementation differences between fread and read.csv?
Vijay
  • 387
  • 2
  • 4
  • 8
27
votes
3 answers

Blank space not recognised as NA in fread

I have a large file which has to be imported in R. I used fread for this purpose. fread is recognizing blank spaces from numeric fields as NA but it is not recognizing blank spaces from character and integer64 fields as NA. fread recognises blank…
SJB
  • 662
  • 7
  • 15
24
votes
2 answers

Speed up odbc::dbFetch

I'm trying to analyze data stored in an SQL database (MS SQL server) in R, and on a mac. Typical queries might return a few GB of data, and the entire database is a few TB. So far, I've been using the R package odbc, and it seems to work pretty…
Michael Roswell
  • 1,300
  • 12
  • 31
22
votes
2 answers

How to check if a PHP stream resource is readable or writable?

In PHP, how do I check if a stream resource (or file pointer, handle, or whatever you want to call them) is either readable or writable? For example, if you're faced with a situation where you know nothing about how the resource was opened or…
jnrbsn
  • 2,498
  • 1
  • 18
  • 25
22
votes
4 answers

Does fread move the file pointer?

Simple question, When i use fread: fread(ArrayA, sizeof(Reg), sizeBlock, fp); My file pointer, fp is moved ahead?
richardaum
  • 6,651
  • 12
  • 48
  • 64
21
votes
10 answers

PHP - Returning the last line in a file?

I'm guessing it's fgets, but I can't find the specific syntax. I'm trying to read out (in a string I'm thinking is easier) the last line added to a log file.
waxical
  • 3,826
  • 8
  • 45
  • 69
20
votes
3 answers

Write and read php object in a text file?

I want to write a php object in a text file. The php object is like that $obj = new stdClass(); $obj->name = "My Name"; $obj->birthdate = "YYYY-MM-DD"; $obj->position = "My position"; I want to write this $obj in a text file. The text file is…
Nantu
  • 557
  • 1
  • 4
  • 16
1
2 3
…
99 100