Can't read file with cyrillic path in C++

Question

I'm trying to read file, which contains Cyrillic characters in their path, and got ifstream.is_open() == false This is my code:

std::string ReadFile(const std::string &path) {
    std::string newLine, fileContent;
    std::ifstream in(path.c_str(), std::ios::in);

    if (!in.is_open()) {
        return std::string("isn't opened");
    }

    while (in.good()) {
        getline(in, newLine);
        fileContent += newLine;
    }

    in.close();

    return fileContent;
}

int main() {
    std::string path = "C:\\test\\документ.txt";
    std::string content = ReadFile(path);
    std::cout << content << std::endl;
    return 0;
}

Specified file exists

I'm trying to find solution in google, but I got nothing

Here is links, which I saw:

I don't need wstring

The same as previous

no answer here

is not about C++

has no answer too

P.S. I need to get file's content in string, not in wstring

THIS IS ENCODING SETTINGS OF MY IDE (CLION 2017.1)

Answer 1

You'll need an up-to-date compiler or Boost. std::filesystem::path can handle these names, but it's new in the C++17 standard. Your compiler may still have it as std::experimental::filesystem::path, or else you'd use the third-party boost::filesystem::path. The interfaces are pretty comparable as the Boost version served as the inspiration.

Answer 2

The definition for std::string is std::basic_string, so your Cyrillic chararecters are not stored as intended. Atleast, try to use std::wstring to store your file path and then you can read from file using std::string.

Answer 3

First of all, set your project settings to use UTF-8 encoding instead of windows-1251. Until standard library gets really good (not any time soon) you basically can not rely on it if you want to deal with io properly. To make input stream read from files on Windows you need to write your own custom input stream buffer that opens files using 2-byte wide chars or rely on some third-party implementations of such routines. Here is some incomplete (but sufficient for your example) implementation:

// assuming that usual Windows SDK macros such as _UNICODE, WIN32_LEAN_AND_MEAN are defined above
#include <Windows.h>

#include <string>
#include <iostream>
#include <system_error>
#include <memory>
#include <utility>
#include <cstdlib>
#include <cstdio>

static_assert(2 == sizeof(wchar_t), "wchar_t size must be 2 bytes");

using namespace ::std;

class MyStreamBuf final: public streambuf
{
    #pragma region Fields
    private: ::HANDLE const  m_file_handle;
    private: char            m_buffer; // typically buffer should be much bigger
    #pragma endregion

    public: explicit
    MyStreamBuf(wchar_t const * psz_file_path)
    :   m_file_handle(::CreateFileW(psz_file_path, FILE_GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL))
    ,   m_buffer{}
    {
        if(INVALID_HANDLE_VALUE == m_file_handle)
        {
            auto const error_code{::GetLastError()};
            throw(system_error(static_cast< int >(error_code), system_category(), "::CreateFileW call failed"));
        }
    }

    public:
    ~MyStreamBuf(void)
    {
        auto const closed{::CloseHandle(m_file_handle)};
        if(FALSE == closed)
        {
            auto const error_code{::GetLastError()};
            //throw(::std::system_error(static_cast< int >(error_code), system_category(), "::CloseHandle call failed"));
            // throwing in destructor is kinda wrong
            // but if CloseHandle returned false then our program is in inconsistent state
            // and must be terminated anyway
            (void) error_code; // not used
            abort();
        }
    }

    private: auto
    underflow(void) -> int_type override
    {
        ::DWORD bytes_count_to_read{1};
        ::DWORD read_bytes_count{};
        {
            auto const succeeded{::ReadFile(m_file_handle, addressof(m_buffer), bytes_count_to_read, addressof(read_bytes_count), nullptr)};
            if(FALSE == succeeded)
            {
                auto const error_code{::GetLastError()};
                setg(nullptr, nullptr, nullptr);
                throw(system_error(static_cast< int >(error_code), system_category(), "::ReadFile call failed"));
            }
        }
        if(0 == read_bytes_count)
        {
            setg(nullptr, nullptr, nullptr);
            return(EOF);
        }
        setg(addressof(m_buffer), addressof(m_buffer), addressof(m_buffer) + 1);
        return(m_buffer);
    }
};

string
MyReadFile(wchar_t const * psz_file_path)
{
    istream in(new MyStreamBuf(psz_file_path)); // note that we create normal stream
    string new_line;
    string file_content;
    while(in.good())
    {
        getline(in, new_line);
        file_content += new_line;
    }
    return(::std::move(file_content));
}

int
main(void)
{
    string content = MyReadFile(L"C:\\test\\документ.txt"); // note that path is a wide string
    cout << content << endl;
    return 0;
}

Answer 4

Change your code to use wstring and save your file using Unicode encoding (non UTF8 one, use USC-2, UTF16 or something like that). MSVC has non-standard overload specifically for this reason to be able to handle non-ascii chars in filenames:

std::string ReadFile(const std::wstring &path)
{
    std::string newLine, fileContent;
    std::ifstream in(path.c_str(), std::ios::in);

    if (!in)
        return std::string("isn't opened");

    while (getline(in, newLine))
        fileContent += newLine;

    return fileContent;
}

int main()
{
    std::wstring path = L"C:\\test\\документ.txt";
    std::string content = ReadFile(path);
    std::cout << content << std::endl;
}

Also, note corrected ReadFile code.