Java how to start reading file after Nth line with good performance

Question

I have a file where data keeps appending. I am using java to read that file and process the data. To get the latest data, I am storing the offset till where I have read the file and continue reading from that offset when java process runs next.

RandomAccessFile f = new RandomAccessFile("file.txt","r");
f.seek(offset)

The problem here is performance. Its around 300 times slower than BufferedReader. Is it possible to resume reading from particular line using BufferedReader?

import java.io.RandomAccessFile;
import java.io.FileReader;
import java.io.IOException;

public class ReadFileExample {

 public static void main(String[] args) {
  RandomAccessFile objReader = null;
  try {
   String strCurrentLine;
   long startTime = System.currentTimeMillis();

   objReader = new RandomAccessFile("auditlog-2018-12-21.txt", "r");

   while ((strCurrentLine = objReader.readLine()) != null) {

   }
   System.out.println(System.currentTimeMillis()-startTime);

  } catch (Exception e) {

   e.printStackTrace();

  } finally {

   try {
    if (objReader != null)
     objReader.close();
   } catch (IOException ex) {
    ex.printStackTrace();
   }
  }
 }
}

Reading 30M file having 100,000 lines takes 12 s while replacing RandomAccessFile with BufferReader takes less than 400ms.

That entirely depends on how you're reading. You shouldn't read byte by byte - only read large blocks of data - 8kb or 16kb are good sizes. This minimizes expensive operating system calls. — Erwin Bolwidt, Mar 21 '19 at 07:14
Show us the code where you read the file. Performance issues can be there as well. — Pavel Smirnov, Mar 21 '19 at 07:29
Possibly helpful: https://stackoverflow.com/q/5614206/3788176 — Andy Turner, Mar 21 '19 at 08:00

score 0 · Answer 1 · answered Mar 21 '19 at 07:46

you can try the below code

try {
    BufferedReader reader = Files.newBufferedReader(Paths.get("file.txt"), StandardCharsets.UTF_8);
    List<String> line = reader.lines().skip(31).limit(1).collect(Collectors.toList());
    line.stream().forEach(System.out::println);
    }catch(Exception e){

    }

Java how to start reading file after Nth line with good performance

1 Answers1