why org.apache.xerces.parsers.SAXParser does not skip BOM in utf8 encoded xml?

Question

I have an xml with utf8 encoding. And this file contains BOM a beginning of the file. So during parsing I am facing with org.xml.sax.SAXParseException: Content is not allowed in prolog. I can not remove those 3 bytes from the files. I can not load file into memory and remove them here (files are big). So for performance reasons I'm using SAX parser and want just to skip those 3 bytes if they are present before "" tag. Should I inherit InputStreamReader for this?

I'm new in java - show me the right way please.

possible duplicate of [Byte order mark screws up file reading in Java](http://stackoverflow.com/questions/1835430/byte-order-mark-screws-up-file-reading-in-java) — Jim Dagg, Oct 07 '13 at 16:56

score 4 · Accepted Answer · edited May 23 '17 at 12:08

4

This has come up before, and I found the answer on Stack Overflow when it happened to me. The linked answer uses a PushbackInputStream to test for the BOM.

edited May 23 '17 at 12:08

Community

1
1

answered Mar 18 '11 at 15:15

Adrian Cox

6,204
5
41
68

score 3 · Answer 2 · answered Mar 18 '11 at 15:17

3

I've experienced the same problem and I've solved it with this code:

private static InputStream checkForUtf8BOM(InputStream inputStream) throws IOException {
    PushbackInputStream pushbackInputStream = new PushbackInputStream(new BufferedInputStream(inputStream), 3);
    byte[] bom = new byte[3];
    if (pushbackInputStream.read(bom) != -1) {
        if (!(bom[0] == (byte) 0xEF && bom[1] == (byte) 0xBB && bom[2] == (byte) 0xBF)) {
            pushbackInputStream.unread(bom);
        }
    }
    return pushbackInputStream;
}

answered Mar 18 '11 at 15:17

javanna

59,145
14
144
125

This is for UTF8... I assume UTF16 would differ (I believe its only 2 bytes)? – Trinition Mar 25 '11 at 19:54
Sorry for the late. Yes, UTF16 has BOM with only two bytes: 0xFE 0xFF (big-endian) or 0xFF 0xFE (little-endian). – javanna Apr 01 '11 at 06:48

score 2 · Answer 3 · answered May 06 '13 at 08:40

private static char[] UTF32BE = { 0x0000, 0xFEFF };
private static char[] UTF32LE = { 0xFFFE, 0x0000 };
private static char[] UTF16BE = { 0xFEFF };
private static char[] UTF16LE = { 0xFFFE };
private static char[] UTF8 = { 0xEFBB, 0xBF };

private static boolean removeBOM(Reader reader, char[] bom) throws Exception {
    int bomLength = bom.length;
    reader.mark(bomLength);
    char[] possibleBOM = new char[bomLength];
    reader.read(possibleBOM);
    for (int x = 0; x < bomLength; x++) {
        if ((int) bom[x] != (int) possibleBOM[x]) {
            reader.reset();
            return false;
        }
    }
    return true;
}

private static void removeBOM(Reader reader) throws Exception {
    if (removeBOM(reader, UTF32BE)) {
        return;
    }
    if (removeBOM(reader, UTF32LE)) {
        return;
    }
    if (removeBOM(reader, UTF16BE)) {
        return;
    }
    if (removeBOM(reader, UTF16LE)) {
        return;
    }
    if (removeBOM(reader, UTF8)) {
        return;
    }
}

usage:

// xml can be read from a file, url or string through a stream
URL url = new URL("some xml url");
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(url.openStream()));
removeBOM(bufferedReader);

why org.apache.xerces.parsers.SAXParser does not skip BOM in utf8 encoded xml?

3 Answers3

Linked