Source.fromResource() works locally, but throws java.nio.charset.MalformedInputException on server

Question

I'm using this code to read a resource:

  val source = Source.fromResource(pathWithoutSlash)
  val lines:Seq[String] = (for (l <- source.getLines() if ! l.trim.isEmpty) yield l.trim).toList

This code works fine when I run it locally - but on the server, it fails with:

Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
    at java.base/java.nio.charset.CoderResult.throwException(CoderResult.java:274)
    at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
    at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
    at java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
    at java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
    at java.base/java.io.BufferedReader.readLine(BufferedReader.java:392)
    at scala.io.BufferedSource$BufferedLineIterator.hasNext(BufferedSource.scala:70)

I'm guessing its because the file does contain some accented characters, like: éclair's , and probably the default charset being used on the server is different from what I have locally.

My question is, how can I change the charset on the server so it matches whatever I have locally (and how can I check what I have locally)?

Thanks.

score 9 · Accepted Answer · answered Apr 13 '19 at 12:57

9

I'd assume that the implicit Codec value that you can see with

println(implicitly[scala.io.Codec])

is different on your server. If I understand it correctly, it should evaluate to scala.io.Codec.fallbackSystemCodec. Just pass the appropriate Codec explicitly (the fromResource method takes an implicit Codec in the second parameter list), e.g.:

val source = Source.fromResource(pathWithoutSlash)(Codec.UTF8)

answered Apr 13 '19 at 12:57

Andrey Tyukin

43,673
4
57
93

Any ideas how the fallback system codec could be changed? Is there a jvm setting? – Ali Apr 14 '19 at 05:26
1

@ClickUpvote From the code, it looks as if it's essentially a wrapper around [`java.nio.charset.Charset.defaultCharset()`](https://docs.oracle.com/javase/7/docs/api/java/nio/charset/Charset.html#defaultCharset()). Maybe [here](https://stackoverflow.com/questions/361975/setting-the-default-java-character-encoding) is something useful. – Andrey Tyukin Apr 14 '19 at 10:01

Source.fromResource() works locally, but throws java.nio.charset.MalformedInputException on server

1 Answers1