I'm trying to run a command on PowerShell with some Persian text but it is not working as expected.
$message = @"
سلام امین
"@
& some.exe @('-m',"$message")
Unfortunately the text that reaches to some.exe is سلام امین which is clearly wrong.
How to fix this?
Update:
Tried this and still not working as expected.
$OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding
$message = @"
سلام امین
"@
Write-Output $message
You can set the output encoding as per this answer.
$OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding
PS> $message = @"
>> سلام امین
>> "@
PS> echoargs $message
Arg 0 is <???? ???? >
Command line:
"C:\Windows\system32\echoargs.exe" "???? ???? "
PS> $OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding
PS> echoargs $message
Arg 0 is <سلام امین >
Command line:
"C:\Windows\system32\echoargs.exe" "سلام امین "
If you choose to put this in your profile, be aware of the possible problems it may cause.
Generally, the @Dabombber answer is correct. but in my case, the problem was that the script file that I have saved on disk was not saved with Unicode encoding. It is strange, cause I used VS Code to write that script and I assumed that VS Code is aware of this.
Anyway, The solution was to open the script with Powershell ISE and save it again. That fixed the problem.
The issue is not in PowerShell but in some.exe. Strings in PowerShell is always in UTF-16 because .NET has only a single type of string that's encoded in UTF-16. When you call some.exe Windows will convert the input parameters to ANSI if you use int main(int argc, char *argv[]). If the Windows code page is not set correctly then you won't be able to get those characters correctly
One solution is to change to int wmain(int argc, wchar_t *argv[], wchar_t *envp[]); to get Unicode parameters. Or if you must use main() for portability then call GetCommandLineW to get the Unicode command line then parse it with CommandLineToArgvW
The better solution is to change to UTF-8 locale. That way it's much easier to port the code to other platforms, and there would be no need to deal with the Unicode/ANSI mess in Windows anymore. Just upgrade the Windows SDK and set the /utf8 flag, along with linking the Windows SDK statically if you need to target older Windows
If changing Windows SDK is not an option then you can use Boost.Nowide and use UTF-8 everywhere. That way argv[] will also be in UTF-8
