4

Since new function 'Beta: Use Unicode UTF-8 for worldwide language support' is added on Windows10, I thought it is possible for R to convert locale environment to UTF-8. However, when I try to change system locale to UTF-8 by

Sys.setlocale(locale = "Japanese_Japan.65001") 

or

Sys.setlocale(locale = "Japanese_Japan.UTF-8") 

I get

In Sys.setlocale("Japanese_Japan.65001") :
OS reports request to set locale to "Japanese_Japan.65001" cannot be honored

For now, does Windows allow R to use UTF-8?

(Because I am not very familiar with locale problem, I welcome comments if there should be more information.)

infomation

> Sys.getlocale()
[1] "LC_COLLATE=Japanese_Japan.932;LC_CTYPE=Japanese_Japan.932;LC_MONETARY=Japanese_Japan.932;LC_NUMERIC=C;LC_TIME=Japanese_Japan.932"
tragoat
  • 63
  • 1
  • 5

3 Answers3

4

UPDATE: The (upcoming) R 4.2.0 should fully support UTF-8 on Windows: https://developer.r-project.org/Blog/public/2021/12/07/upcoming-changes-in-r-4.2-on-windows/


It appears that R has built experimental binaries that fully support UTF-8 on Windows 10, but since the project was marked as "experimental" as of 2020-07-30 and the official conclusion was:

Based also on this experience, I believe that switching to UCRT is already possible and I expect that building a complete toolchain should take a small number of months. It is I think the only realistic way to support Unicode characters (not representable in native encoding) reliably in R on Windows.

It clearly means that full UTF-8 support in R on Windows is still a plan for a bit more distant future.

Source: https://developer.r-project.org/Blog/public/2020/07/30/windows/utf-8-build-of-r-and-cran-packages/index.html

Martin Modrák
  • 746
  • 8
  • 17
2

Sys.setlocale(locale = foo) defaults to category = "LC_ALL"; maybe set aspects of the locale for the R process individually, e.g. as follows:

locales <- c("LC_COLLATE","LC_CTYPE","LC_MONETARY","LC_NUMERIC","LC_TIME");
for (x in locales) { Sys.setlocale(category = x, locale="Japanese_Japan.65001")}

Please observe all warnings from above code snippet and further notes from locales: Query or Set Aspects of the Locale article:

  • Attempts to change the character set (by Sys.setlocale("LC_CTYPE", ), if that implies a different character set) during a session may not work and are likely to lead to some confusion.
  • Setting "LC_NUMERIC" to any value other than "C" may cause R to function anomalously, so gives a warning.
  • Almost all the output routines used by R itself under Windows ignore the setting of "LC_NUMERIC" since they make use of the Trio library which is not internationalized.

For instance, my locale is Czech so I tried the following code snippet (itemized above loop to see the results and warnings in sequence):

Sys.getlocale(category = "LC_ALL")
Sys.setlocale(category = "LC_COLLATE" , locale="Czech_Czechia.65001")
Sys.setlocale(category = "LC_CTYPE"   , locale="Czech_Czechia.65001")
Sys.setlocale(category = "LC_MONETARY", locale="Czech_Czechia.65001")
Sys.setlocale(category = "LC_NUMERIC" , locale="Czech_Czechia.65001")
Sys.setlocale(category = "LC_TIME"    , locale="Czech_Czechia.65001")
Sys.getlocale(category = "LC_ALL")

Output (pasted into the RStudio console):

> Sys.getlocale()
[1] "LC_COLLATE=Czech_Czechia.1250;LC_CTYPE=Czech_Czechia.1250;LC_MONETARY=Czech_Czechia.1250;LC_NUMERIC=C;LC_TIME=Czech_Czechia.1250"
> Sys.setlocale(category = "LC_COLLATE" , locale="Czech_Czechia.65001")
[1] "Czech_Czechia.65001"
> Sys.setlocale(category = "LC_CTYPE"   , locale="Czech_Czechia.65001")
[1] ""
Warning message:
In Sys.setlocale(category = "LC_CTYPE", locale = "Czech_Czechia.65001") :
  OS reports request to set locale to "Czech_Czechia.65001" cannot be honored
> Sys.setlocale(category = "LC_MONETARY", locale="Czech_Czechia.65001")
[1] "Czech_Czechia.65001"
> Sys.setlocale(category = "LC_NUMERIC" , locale="Czech_Czechia.65001")
[1] "Czech_Czechia.65001"
Warning message:
In Sys.setlocale(category = "LC_NUMERIC", locale = "Czech_Czechia.65001") :
  setting 'LC_NUMERIC' may cause R to function strangely
> Sys.setlocale(category = "LC_TIME"    , locale="Czech_Czechia.65001")
[1] "Czech_Czechia.65001"
> Sys.getlocale(category = "LC_ALL")
[1] "LC_COLLATE=Czech_Czechia.65001;LC_CTYPE=Czech_Czechia.1250;LC_MONETARY=Czech_Czechia.65001;LC_NUMERIC=Czech_Czechia.65001;LC_TIME=Czech_Czechia.65001"
> 
JosefZ
  • 28,460
  • 5
  • 44
  • 83
  • 1
    Thank you for answering. I tried to change each elements of locales, but I get the same refusal above every time. I observe the locale problem of Windows annoys advanced R users in Japan, so it seems that this problem is out of control to me. Thank you again. –  tragoat Jul 05 '20 at 13:39
0

The best way to use R in Windows to this day (August 22nd, 2020) is to install WSL 2 (Windows Subsystem for Linux) and connect to RStudio Server via a web browser.

Instructions:

xwhitelight
  • 1,569
  • 1
  • 10
  • 19