read file of unknown encoding c++

Question

I want to read the contents of log file which is generated by msiexec. The encoding type of the log file is UCS-2 LE BOM(not sure how this encoding type used while generating log file).

When I read the content of this file using below code, I am getting non ascii characters in the string.

std::string errMsg;
std::ifstream ifs("install.log");
for (std::string line; std::getline(ifs, line); /**/)
{
    errMsg.append(line);
}

Is there any way to read a file of any encoding and convert to ANSI using C++17?

`of unknown encoding` `encoding type of the log file is UCS-2 LE BOM` so it's unknown or known? `Is there any way to read a file of any encoding and convert to ANSI` if ANSI refers to windows-1252, then it depends, because there are (many) characters that can't be represented in ansi encoding. Either way, you have to know the source encoding anyway. And anyway, I suspect you just want to use wide stream functions. — KamilCuk, Nov 16 '20 at 10:23
Does this answer your question? [How to read a UCS-2 file?](https://stackoverflow.com/questions/11643500/how-to-read-a-ucs-2-file) — , Nov 16 '20 at 10:27

eerorika · Answer 1 · 2020-11-16T10:47:07.833

read file of unknown encoding c++

You can read any data if you use binary mode. Binary mode will not treat null character as end of string and doesn't transform end of line sequences from system specific formats.

and convert to ANSI

This is like asking how to translate a language that you don't understand. It's not possible in general. There is no way to convert unknown encoding to another encoding. You must know the source encoding to achieve that. You also need to know the destination encoding. "ANSI" is not an encoding.

Source encoding could in theory be guessed based on the content, but specifying rules for such detection would be quite difficult. It would also likely be a highly error prone guess unless the input is long and happens to contain distinguishing special characters in typical arrangement. I hypothesise that a neural network could be trained to guess encodings. C++17 has no standard API for training or querying neural networks.

read file of unknown encoding c++

1 Answers1