Determine file encoding and print umlaute

Question

I have a txt file. I'm not sure of the encoding of this file. Probably it's s EBCDIC. I have problem with Umlaute (äöü, ÜÄÖ) e.g. For example: Displayed: Mnchen should be: München Url to test file: http://wyslijto.pl/plik/yiewa11y3p

java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;

public class Main {

    public static void main(String[] args) throws IOException {
        BufferedReader in = new BufferedReader(
                new InputStreamReader(
                        new FileInputStream("/Downloads/test.txt")));

//        BufferedReader in = new BufferedReader(
//                new InputStreamReader(
//                        new FileInputStream("/Downloads/test.txt"), Charset.forName("windows-1252")));
        String str;
        while ((str = in.readLine()) != null) {
            System.out.println(str);
        }
    }
}

Try to use a proper encoding and set the buffereader with UTF8 — curiouscupcake, Mar 13 '19 at 21:51
Basically, you need to find out the encoding. I doubt that it's EBCDIC, or it would be considerably more broken than it is now. I suggest you find out more about what produced the file, and use that information to try to find out the encoding. — Jon Skeet, Mar 13 '19 at 22:06
`file` says: "Downloads/test.txt: UTF-8 Unicode (with BOM) text, with no line terminators", so @vs97 's answer should work. — Robert, Mar 13 '19 at 22:07

vs97 · Answer 1 · 2019-03-13T22:15:20.543

1

Unfortunately there is no certain way to detect encoding without knowing what was used to create the file in the first place. I will refer you to this question, it has a lot of suggestions how one could make an intelligent guess what the encoding really is.

Once you know the encoding (that's the difficult part), its simple. For example, if the encoding was to be UTF-8, use UTF-8 charset with your InputStreamReader:

BufferedReader in = new BufferedReader(
                new InputStreamReader(
                        new FileInputStream("/Downloads/test.txt"), StandardCharsets.UTF_8));

In general, the supported Charsets are:

ISO_8859_1
US_ASCII
UTF_16
UTF_16BE
UTF_16LE
UTF_8

edited Mar 13 '19 at 22:15

answered Mar 13 '19 at 22:02

vs97

5,765
3
28
41

1

That will only help if it actually *is* UTF-8. The OP says they don't know what the encoding is. – Jon Skeet Mar 13 '19 at 22:05
(And `Files.newBufferedReader` would be a simpler way of doing the above, as that uses UTF-8 by default.) – Jon Skeet Mar 13 '19 at 22:07
The problem it is that the character is UTF-8. Probably double encoded, so it require two decoding. – Giacomo Catenazzi Mar 14 '19 at 09:48

Determine file encoding and print umlaute

1 Answers1