How read Japanese fields from CSV file into java beans?

Question

I've tried several popular CSV to java deserializers - OpenCSV, JSefa, and Smooks - none correctly read the file:

First Name,Last Name
エリック,山中
花子,鈴木
一郎,鈴木
裕子,田中
政治,山村

into my java object collection.

OpenCsv code:

    HeaderColumnNameTranslateMappingStrategy<Contact> strat = new HeaderColumnNameTranslateMappingStrategy<Contact>();
    strat.setType(Contact.class);
    strat.setColumnMapping(colNameTranslateMap);
    InputStreamReader fileReader=null;
    CsvToBean<Contact> csv = new CsvToBean<Contact>();
    fileReader = new InputStreamReader(new FileInputStream(file), "UTF-8");

    contacts = csv.parse(strat, new CSVReader(fileReader));

I've tried setting the Charset to UTF-8, UTF-16 and ISO-8859-1 when I create the FileInputStream, but the collection is never populated properly. As seen in the debugger and System.out the fields contain garbage and often the number of records is wrong.

How in the world did you specify a charset when creating a `FileInputStream`? — Matt Ball, Nov 03 '11 at 23:58

score 4 · Accepted Answer · answered Nov 03 '11 at 23:52

4

FileInputStream is for reading streams of binary data, like an mp3 or PNG. Instead of a FIS, use a FileReader for reading streams of characters.
To be blunt: who cares what charsets you tried using if they didn't work? You need to figure out what encoding the CSV file is actually using, and set that encoding when reading the file. To specify the encoding when using a FileReader:

The constructors of this class assume that the default character encoding and the default byte-buffer size are appropriate. To specify these values yourself, construct an InputStreamReader on a FileInputStream.

answered Nov 03 '11 at 23:52

Matt Ball

354,903
100
647
710

FileReader didn't work either. The problem w/ FileReader is it uses the JVM's default charset, you can't set it the way you can w/ FileInputStream. – MebAlone Nov 03 '11 at 23:58
Let's see some code. Please edit your question to include an [SSCCE](http://sscce.org). – Matt Ball Nov 03 '11 at 23:59
Please read my answer carefully, specifically the bolded part of the quote. – Matt Ball Nov 04 '11 at 00:00
I'll edit the question but in all cases I am doing exactly what you suggested. – MebAlone Nov 04 '11 at 00:04
Thanks `:)` BTW, if you know (or can find out) what the actual encoding of the CSV file is, that will help. A lot. – Matt Ball Nov 04 '11 at 00:06
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/4705/discussion-between-matt-ball-and-mebalone) – Matt Ball Nov 04 '11 at 00:18
When I open a FileReader on the CSV and call getEncoding() it returns UTF8. – MebAlone Nov 04 '11 at 00:18

How read Japanese fields from CSV file into java beans?

1 Answers1

Linked