How to Pass Unicode Arguments to G++?

Question

I am trying to use G++ to compile a simple C++ program. I am running Windows 10 and I have installed MinGW. So I tried to compile this file C:\Users\Vesk\Desktop\Информатика\Hello World.cpp with G++ by typing g++ "C:\Users\Vesk\Desktop\Информатика\Hello World.cpp" -o "C:\Users\Vesk\Desktop\Информатика\Hello World.exe" in the Command Prompt. G++ though didn't compile the file and gave me this error message:

g++: error: C:\Users\Vesk\Desktop\???????????\Hello World.cpp: Invalid argument
g++: fatal error: no input files
compilation terminated.

'Информатика' is just a word written in Cyrillic, so I was confused what the problem was. But then I just renamed the 'Информатика' folder to 'Informatics'. I tried to compile the file again with g++ "C:\Users\Vesk\Desktop\Informatics\Hello World.cpp" -o "C:\Users\Vesk\Desktop\Informatics\Hello World.exe". And lo and behold it worked. G++ compiled the file and the executable was there in the folder and working. But is there any way to actually compile a file if its path contains Cyrillic (or other Unicode) characters? If so, how?

Answer 1

Windows uses UTF-16 for Unicode file names. To my knowledge, it does not support UTF-8 as a locale although that would be very useful.

I tried on a very old MinGW G++ 4.6.3 and indeed it does not support Unicode characters in file paths that are outside current locale. Don't know about more recent MinGW GCC. A first possible solution would be to use a Russian locale.

For a Windows application to properly support Unicode file names, it needs to handle paths as wchar_t wide characters. The int main(int argc, const char* argv[]) classical signature for example must be replaced by int wmain(int argc, const wchar_t* argv[]). For a portable software like GCC, this is a complication that may not be worth it. Extremely few people will put characters in source file paths that are outside their current locale.

I tried G++ 10.2.0 on Cygwin and it works. This is because all Cygwin software link with cygwin1.dll which, among other services, automatically convert all UTF-8 paths to UTF-16.

Answer 2

You should first get the command line with UTF16 encoding with GetCommandLineW function (https://learn.microsoft.com/en-us/windows/win32/api/processenv/nf-processenv-getcommandlinew) and then separate the tokens with CommandLineToArgW (https://learn.microsoft.com/en-us/windows/win32/api/shellapi/nf-shellapi-commandlinetoargvw).

If you want UTF8 encoded strings you need to convert them, a simple, open source and useful tool to convert strings with different encodings in C++20 can be found here.