17

I am getting this error:

io.MalformedByteSequenceException: Invalid byte 2 of 2-byte UTF-8 sequence

The solution is to read and write a file in UTF-8.

My code is:

InputStream input = null;
OutputStream output = null;
OutputStreamWriter bufferedWriter = new OutputStreamWriter( output, "UTF8");
input = new URL(url).openStream();
output = new FileOutputStream("DirectionResponse.xml");
byte[] buffer = new byte[1024];
for (int length = 0; (length = input.read(buffer)) > 0;) {
   output.write(buffer, 0, length);
}
BufferedReader br = new BufferedReader(new FileReader("DirectionResponse.xml" ));
FileWriter fstream = new FileWriter("ppre_DirectionResponse.xml");
BufferedWriter out = new BufferedWriter(fstream);

I'm reading a url and writing it to a file DirectionResponse.xml. Then reading DirectionResponse.xml and writing the same as ppre_DirecionResponse.xml for processing.

How do I change this so that reading and writing is done in UTF-8?

Ola Ström
  • 4,136
  • 5
  • 22
  • 41
Gaurav Wadhwani
  • 358
  • 2
  • 6
  • 16

2 Answers2

37

First, you need to call output.close() (or at least call output.flush()) before you reopen the file for input. That's probably the main cause of your problems.

Then, you shouldn't use FileReader or FileWriter for this because it always uses the platform-default encoding (which is often not UTF-8). From the docs for FileReader:

The constructors of this class assume that the default character encoding and the default byte-buffer size are appropriate.

You have the same problem when using a FileWriter. Replace this:

BufferedReader br = new BufferedReader(new FileReader("DirectionResponse.xml" ));

with something like this:

BufferedReader br = new BufferedReader(new InputStreamReader(
    new FileInputStream("DirectionResponse.xml"), "UTF-8"));

and similarly for fstream.

Ted Hopp
  • 232,168
  • 48
  • 399
  • 521
  • 1
    @Aubin - Sure, at least if you're talking about the input stream: `URLConnection conn = url.openConnection(); InputStream is = conn.getInputStream();`. Then use `is` as the input stream. – Ted Hopp Nov 12 '12 at 20:14
  • @user905911 - I noted another problem with your code. See the first paragraph of my revised answer. – Ted Hopp Nov 12 '12 at 20:19
  • @TedHopp : Sir, it worked. But i dont no how to change FileWriter. Should i change to OutputStreamWriter ? – Gaurav Wadhwani Nov 12 '12 at 20:48
  • ok so i changed that to FileOutputStream fos = new FileOutputStream("ppre_DirectionResponse.xml"); Writer out = new OutputStreamWriter(fos, "UTF8"); – Gaurav Wadhwani Nov 12 '12 at 20:54
  • Since Java 7 it could be done in a shorter way: `BufferedReader br = Files.newBufferedReader(Paths.get("DirectionResponse.xml"), StandardCharsets.UTF_8);` – Vladimir Vagaytsev Jul 11 '16 at 19:21
2

Read and Write UTF-8 File in Java

I see you are writing in utf-8 but not specifically reading in utf-8. Follow the example I've provided in the link.

try {
   Reader reader =
      new InputStreamReader(
         new FileInputStream(args[0]),"UTF-8");
   BufferedReader fin = new BufferedReader(reader);
   Writer writer =
      new OutputStreamWriter(
         new FileOutputStream(args[1]), "UTF-8");
   BufferedWriter fout = new BufferedWriter(writer);
   String s;
   while ((s=fin.readLine())!=null) {
      fout.write(s);
      fout.newLine();
   }

            //Remember to call close. 
            //calling close on a BufferedReader/BufferedWriter 
            // will automatically call close on its underlying stream 
   fin.close();
   fout.close();
} catch (IOException e) {
   e.printStackTrace();
}
Aubin
  • 14,617
  • 9
  • 61
  • 84
Aravind Yarram
  • 78,777
  • 46
  • 231
  • 327