Different Number of Character in Java Android InputStream and C++ ifstream

Question

So, I am developing android application that read JSON text file containing some data. I have a 300 kb (307,312 bytes) JSON in a text file (here). I also develop desktop application (cpp) to generate and loading (and parsing) the JSON text file.

When I try to open and read it using ifstream in c++, I get the string length correctly (307,312). I even succesfully parse it.

Here is my code in C++:

std::string json = "";
std::string line;
std::ifstream myfile(textfile.txt);

if(myfile.is_open()){
    while(std::getline(myfile, line)){
        json += line;
        json.push_back('\n');
    }
    json.pop_back(); // pop back the last '\n'
    myfile.close();
}else{
    std::cout << "Unable to open file";
}

In my android application, I put my JSON text file in res/raw folder. When I try to open and read using InputStream, the length of the string only 291,896. And I can't parse it (I parse it using jni with the same c++ code, maybe it is not important).

InputStream is = getResources().openRawResource(R.raw.textfile);
byte[] b = new byte[is.available()];
is.read(b);
in_str = new String(b);

UPDATE:

I also have try using this way.

InputStream is = getResources().openRawResource(R.raw.textfile);
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
String line = reader.readLine();
while(line != null){
    in_str += line;
    in_str += '\n';
    line = reader.readLine();
}
if (in_str != null && in_str.length() > 0) {
    in_str = in_str.substring(0, in_str.length()-1);
}

Even, I tried moving it from res/raw folder to assets folder in java android project. And of course I change the InputStream line to InputStream is = getAssets().open("textfile.txt"). Still not working.

score 0 · Answer 1 · edited May 23 '17 at 12:30

Okay, I found the solution. It is the ASCII and UTF-8 problem.

From here:

UTF-8 Variable length encoding, 1-4 bytes per code point. ASCII values are encoded as ASCII using 1 byte.
ASCII Single byte encoding

My filesize is 307,312 bytes and basically I need to take the character each byte. So, I should need to encode the file as ASCII.

When I am using C++ ifstream, the string size is 307,312. (same as of the number character if it is using ASCII encoding)

Meanwhile, when I am using Java InputStream, the string size is 291,896. I assume that it happens because of the reader is using UTF-8 encoding instead.

So, how to use get ASCII encoding in Java?

Through this thread and this article, we can use InputStreamReader in Java and set it to ASCII. Here is my complete code:

String in_str = "";
try{
    InputStream is = getResources().openRawResource(R.raw.textfile);
    BufferedReader reader = new BufferedReader(new InputStreamReader(is, "ASCII"));
    String line = reader.readLine();
    while(line != null){
        in_str += line;
        in_str += '\n';
        line = reader.readLine();
    }
    if (in_str != null && in_str.length() > 0) {
        in_str = in_str.substring(0, in_str.length()-1);
    }
}catch(Exception e){
    e.printStackTrace();
}

If you have the same problem, hope this helps. Cheers.

Different Number of Character in Java Android InputStream and C++ ifstream

1 Answers1