How to return a line iterator on a big text file in Scala

Question

I have a gigantic 2GB text file which I want to read line by line without pushing everything to memory. I have written the following code snippet for that but it doesn't seem to work at yield call. How should I return the line iterator from a function then?

import java.io.{BufferedReader, File, FileInputStream, InputStream, InputStreamReader}
import java.nio.charset.StandardCharsets

def readLines(fileName: String): Iterable[String] = {
    val bufferedReader = new BufferedReader(new InputStreamReader(getClass.getClassLoader.getResourceAsStream(fileName), StandardCharsets.UTF_8))
    for (line <- bufferedReader.readLine()) {
        yield line
    }
}

Possible duplicate of [How do I read a large CSV file with Scala Stream class?](https://stackoverflow.com/questions/4255021/how-do-i-read-a-large-csv-file-with-scala-stream-class) — joel, Sep 27 '18 at 13:55

Aleksey Isachenkov · Accepted Answer · 2018-09-27T13:57:10.037

4

You should use scala.io.Source

Source.fromFile(fileName).getLines()

Or you can check scala.io.BufferedSource.BufferedLineIterator code and write the same.

edited Sep 27 '18 at 13:57

answered Sep 27 '18 at 13:48

Aleksey Isachenkov

1,230
6
16

How to return a line iterator on a big text file in Scala

1 Answers1