Scala: Read a file line by line to list recursively instead of loop

Question

I come from C++, and I'm trying to use more "functional" style, and decided to switch out my file-reading loop with a file reading recursive function (is that the right term?), and this is what I came up with:

 def readFileToList(list: List[String]) : List[String] = list match {
    case List(head) => list
    case head::tail => fromFile("filename").getLines :: list
    case _ => list
  }

So of course this doesn't work and I'm not exactly sure why. First thing I see is that getLines returns an iterator. So how would one do this in a "functional" way?

Pascal Bugnion · Accepted Answer · 2015-06-28T20:33:59.150

I think you are over-complicating things: getLines does returns an iterator, as you said. You can materialise an iterator into a list using its .toList method:

Source.fromFile("myfile.txt").getLines.toList

As an aside, what if the .toList method didn't exist? You could then use a recursive function to convert an iterator to a list (if you really wanted to -- a simple reduce would be simpler and probably faster):

def materialize[T](it:Iterator[T]):List[T] = 
    materializeHelper(it, List.empty[T])

def materializeHelper[T](it:Iterator[T], accumulator:List[T]):List[T] =
    if(!it.hasNext) {
        accumulator.reverse  
    }
    else {
        materializeHelper(it, it.next :: accumulator)
    }

In general, you should prefer using higher-order functions to recursion or iteration whenever possible. Higher-order functions minimise your exposure to moving parts, which is where bugs get introduced. Consider the same code implemented with foldLeft:

def materialize[T](it:Iterator[T]):List[T] =
  it.foldLeft(List.empty[T]) { 
    (accumulator, elem) => elem :: accumulator 
  }.reverse

This is less error-prone: you don't have to deal with advancing the iterator directly, or with boundary conditions when the iterator is empty.

Would the `Source.fromFile("myfile.txt").getLines.toList` also `close()` the open resource? Last time I checked it doesn't. This is fine for Unit Tests and "dirty" Scripts. However, if you are running this on a production system with long running JVMs I feel this will cause troubles. — marios, Jun 28 '15 at 17:41
You are quite right that it doesn't close the file. One normally writes `val f = Source.fromFile("myfile.txt") ; val lines = f.getLines.toList ; f.close`. Alternatively, it's quite easy to write a function that uses the [loan pattern](http://stackoverflow.com/questions/20762240/loan-pattern-in-scala) to provide self-closing sources. — Pascal Bugnion, Jun 28 '15 at 20:19

score 1 · Answer 2 · answered Jun 28 '15 at 14:40

So, your solution here is kind of an odd hybrid. Functional programming doesn't mean adding boilerplate of a certain form or eschewing reuse. If you are going to use the function Source.fromFile(...), which does the work you might do in a recursive function internally, then just use it, in the way that Pascal Bugnion suggests. If you want, as an exercise, to understand how "internal iteration" might be implemented functionally with recursion rather than a loop, then skip the fromFile(...) method and implement it yourself.

This might look quite similar to Pascal Bugnion's materialize implementation:

import java.io.{BufferedReader, FileReader, File}
import scala.annotation.tailrec

def linesFromFile( file : File ) : List[String] = {

   // stealing from Pascal Bugnion, slightly different style
   def materialize( br : BufferedReader ) : List[String] = materializeReverse( br, Nil ).reverse

   @tailrec
   def materializeReverse( br : BufferedReader, accumulator : List[String] ) : List[String] = {
     br.readLine match {
       case null => accumulator
       case line => materializeReverse( br, line :: accumulator )
     }
   }

   // presuming the default character encoding is good enough
   val br = new BufferedReader( new FileReader( file ) )
   try materialize( br ) finally br.close
}

Scala: Read a file line by line to list recursively instead of loop

2 Answers2