Write UTF-8 files from R

Question

Whereas R seems to handle Unicode characters well internally, I'm not able to output a data frame in R with such UTF-8 Unicode characters. Is there any way to force this?

data.frame(c("hīersumian","ǣmettigan"))->test
write.table(test,"test.txt",row.names=F,col.names=F,quote=F,fileEncoding="UTF-8")

The output text file reads:

hiersumian <U+01E3>mettigan

I am using R version 3.0.2 in a Windows environment (Windows 7).

EDIT

---

It's been suggested in the answers that R is writing the file correctly in UTF-8, and that the problem lies with the software I'm using to view the file. Here's some code where I'm doing everything in R. I'm reading in a text file encoded in UTF-8, and R reads it correctly. Then R writes the file out in UTF-8 and reads it back in again, and now the correct Unicode characters are gone.

read.table("myinputfile.txt",encoding="UTF-8")->myinputfile
myinputfile[1,1]
write.table(myinputfile,"myoutputfile.txt",row.names=F,col.names=F,quote=F,fileEncoding="UTF-8")
read.table("myoutputfile.txt",encoding="UTF-8")->myoutputfile
myoutputfile[1,1]

Console output:

> read.table("myinputfile.txt",encoding="UTF-8")->myinputfile
> myinputfile[1,1]
[1] hīersumian
Levels: hīersumian ǣmettigan
> write.table(myinputfile,"myoutputfile.txt",row.names=F,col.names=F,quote=F,fileEncoding="UTF-8")
> read.table("myoutputfile.txt",encoding="UTF-8")->myoutputfile
> myoutputfile[1,1]
[1] <U+FEFF>hiersumian
Levels: <U+01E3>mettigan <U+FEFF>hiersumian
> 

Answer 1

This "answer" serves rather the purpose of clarifying that there is something odd going on behind the scenes:

"hīersumian" doesn't even make it into the data frame it seems. The "ī"-symbol is in all cases converted to "i".

options("encoding" = "native.enc")
t1 <- data.frame(a = c("hīersumian "), stringsAsFactors=F)
t1
#             a
# 1 hiersumian 

options("encoding" = "UTF-8")
t1 <- data.frame(a = c("hīersumian "), stringsAsFactors=F)
t1
#             a
# 1 hiersumian 

options("encoding" = "UTF-16")
t1 <- data.frame(a = c("hīersumian "), stringsAsFactors=F)
t1
#             a
# 1 hiersumian 

The following sequence successfully writes "ǣmettigan" to the text file:

t2 <- data.frame(a = c("ǣmettigan"), stringsAsFactors=F)

getOption("encoding")
# [1] "native.enc"

Encoding(t2[,"a"]) <- "UTF-16"

write.table(t2,"test.txt",row.names=F,col.names=F,quote=F)

It is not going to work with "encoding" as "UTF-8" or "UTF-16" and also specifying "fileEncoding" will either lead to a defect or no output.

Somewhat disappointing as so far I managed to get all Unicode issues fixed somehow.

Answer 2

I may be missing something OS-specific, but data.table appears to have no problem with this (or perhaps more likely it's an update to R internals since this question was originally posed):

t1 = data.table(a = c("hīersumian", "ǣmettigan"))
tmp = tempfile()
fwrite(t1, tmp)
system(paste('cat', tmp))
# a
# hīersumian
# ǣmettigan
fread(tmp)
#             a
# 1: hīersumian
# 2:  ǣmettigan

Answer 3

I found a blog post that basically says its windows way of encoding text. Lots more detail in post. User should write the file in binary using

writeBin(charToRaw(x), con, endian="little")

https://tomizonor.wordpress.com/2013/04/17/file-utf8-windows/