java.nio.BufferUnderflowException when processing files in Scala

Question

I got a similar problem to this guy while processing 4MB log file. Actually I'm processing multiple files simultaneously but since I keep getting this exception, I decide to just test it for a single file:

val temp = Source.fromFile("./datasource/input.txt")
val dummy = new PrintWriter("test.txt")
var itr = 0
println("Default Buffer size: " + Source.DefaultBufSize)
try {
    for( chr <- temp) {
        dummy.print(chr.toChar)
        itr += 1
        if(itr == 75703) println("Passed line 85")
        if(itr % 256 == 0){ print("..." + itr); temp.reset; System.gc; }
        if(itr == 75703) println("Passed line 87")
        if(itr % 2048 == 0) println("")
        if(itr == 75703) println("Passed line 89")
    }
} finally {
    println("\nFalied at itr = " + itr)
}

What I always get is that it will fails at itr = 75703, while my output file will always be 64KB (65536 Bytes exact). No matter where I put temp.reset or System.gc, all experiments ends up the same.

It seems like the problem relies on some memory allocation but I cannot find any useful information on this problem. Any idea on how to solve this one?

All your helps are greatly appreciated

EDIT: Actually I want to process it as binary files, so this technique is not a good solution, many had recommend me to use BufferedInputStream instead.

Searching, I found http://lampsvn.epfl.ch/trac/scala/ticket/1883. There's also a duplicate of it. Basically, it seems Java and Scala are disagreeing about the encoding on the file, which ends up with such underflows. Not sure what can be done about it. — Daniel C. Sobral, Oct 26 '09 at 11:19
Yeah. According to the ticket page the only real solution to BufferUnderFlow for Scala user is just waiting for Scala 2.8. — Ekkmanz, Oct 26 '09 at 15:34

score 1 · Accepted Answer · edited May 23 '17 at 12:02

1

Why are you calling reset on the Source before it has finished iterating thru the file?

val temp = Source.fromFile("./datasource/input.txt")
try {
  for (line <- tem p.getLines) {
    //whatever
  }
finally temp.reset

Should work just fine with no underflows. See also this question

edited May 23 '17 at 12:02

Community

1
1

answered Oct 26 '09 at 09:32

oxbow_lakes

133,303
56
317
449

java.nio.BufferUnderflowException when processing files in Scala

1 Answers1