How to import last 100 rows using read.csv() in R

Question

Hi I've a huge file and i want to import only the last 100 rows from that file. How can we do that using read.csv() or any alternative?

If you're concerned about speed, then try `fread` from "data.table" and then just extract the rows you need. Similarly, you can use `sqldf`. If you're on a Unix system, you have access to the `tail` command that could be useful. — A5C1D2H2I1M1N2O1R2T1, Aug 30 '13 at 06:37
I know we can use os specific commands but i'm looking for a work around in R itself! — Prasun Velayudhan, Aug 30 '13 at 07:36

score 23 · Accepted Answer · answered Aug 30 '13 at 16:41

23

The package R.utils has a function called countLines(). You could do:

l2keep <- 10
nL <- countLines("your.csv")
df <- read.csv("your.csv", header=FALSE, skip=nL-l2keep)

answered Aug 30 '13 at 16:41

lauratboyer

334
1
4

score 3 · Answer 2 · edited May 23 '17 at 11:52

3

If you are on a *nix system, you are better off using the tail -n 100 command to take the last 100 rows. Anything implemented in R would be slower and potentially much slower is your file is truly huge.

If you are using Windows, you may want to take a look at this SO question.

edited May 23 '17 at 11:52

Community

1
1

answered Aug 30 '13 at 06:45

ktdrv

3,602
3
30
45

ya that's true. So what you are telling is that using some windows function to get the last 100 rows put it in a file and then import into R? – Prasun Velayudhan Aug 30 '13 at 07:27
Pretty much. You can do `seek()` and other "fancy" stuff in R but good luck finding something that is as fast or as simple. – ktdrv Aug 30 '13 at 16:36

balping · Answer 3 · 2018-06-27T16:25:37.660

2

Improvement on @lauratboyer's answer if you want to include headers too:

# read headers only
column_names <- as.vector(t(read.csv("your.csv", header=FALSE, colClasses='character', nrows=1)))

# then last n lines
l2keep <- 10
nL <- R.utils::countLines("your.csv")
df <- read.csv("your.csv", header=FALSE, col.names=column_names, skip=nL-l2keep)

edited Jun 27 '18 at 16:25

answered Jun 26 '18 at 15:29

balping

7,518
3
21
35

score 1 · Answer 4 · answered Aug 30 '13 at 06:48

1

You could use the nrows and skip arguments in read.csv. E.g. if you have a file with 10000 rows and you would only like to import the last 100 rows you could try this:

read.csv("yourfile.csv",nrows=100,skip=9900)

But if it is speed you want, you're probably better off with the solutions given by @Ananda Mahto and @ktdrv

answered Aug 30 '13 at 06:48

Rob

1,460
2
16
23

thanks for the reply.But the problem is my file size is so huge that i'm unable to determine the total number of rows records. – Prasun Velayudhan Aug 30 '13 at 07:26

score 0 · Answer 5 · answered Oct 08 '17 at 18:27

The quick and dirty way that works for me - use fread to read large files while setting select = 1 so that only the first column is read. Then use fread again to read data from the desired rows. Fread is much faster than read.csv or other similar variants. More on fread vs read.csv here: Reason behind speed of fread in data.table package in R

score 0 · Answer 6 · answered Jan 03 '19 at 12:35

0

Read file, use tail function a<-read.csv('c:/..') tail(a,100L)

answered Jan 03 '19 at 12:35

h612

544
2
11

B K · Answer 7 · 2017-11-05T08:33:40.197

-2

give appropriate skip parameter in read.csv()

edited Nov 05 '17 at 08:33

answered Jun 10 '15 at 16:52

B K

723
8
17

1

this doesn't answer the OP's question. They want to *read from a file* only the last 100 rows. Your answer assumes the data set has already been read. – Ben Bolker Jun 10 '15 at 16:55
This reply is useless. It's just a bad copy of @Rob's reply (who replied first). For the record, I am not criticizing Rob's answer, but only this B K's reply – Bruce_Warrior May 08 '23 at 04:58

How to import last 100 rows using read.csv() in R

7 Answers7

Linked