4

I am attempting to read csv files generated by cognos 8 into r using readr.

Example file attached for reproducibility: Example csv file

The following python code works:

df = pd.read_table('csv_test.csv', encoding = 'utf-16')

I've tried the following in R, and none of them return the correct result. They either error (Incomplete multibyte sequence) or read in improperly (as nested lists or similar)

csv_data <- read_table('csv_test.csv')
csv_data <- read_table('csv_test.csv', locale = locale(encoding = 'UTF-16LE'))
csv_data <- read_tsv('csv_test.csv')
csv_data <- read_tsv('csv_test.csv', locale = locale(encoding = 'UTF-16LE'))

I used guess_encoding() to get the UTF-16LE, I also tried UTF-16.

d.ellis
  • 61
  • 6
  • 2
    There's this [open issue for the readr package](https://github.com/tidyverse/readr/issues/306). – Gregor Thomas Dec 28 '17 at 14:51
  • And while there's not a good solution there, [suggested duplicate: Fast method to read csv with UTF-16LE encoding](https://stackoverflow.com/q/36862340/903061) – Gregor Thomas Dec 28 '17 at 14:52
  • Thank you @gregor . I tried the base read.csv file but didn't get that to work. Read_delim as shown in the open github issue will work for now. – d.ellis Dec 28 '17 at 14:59

1 Answers1

2

As mentioned in comment by Gregor, there is an open issue with the readr package for this.

As a workaround, base package read_delim will work:

csv_data <- read.delim('csv_test.csv', stringsAsFactors = FALSE, fileEncoding = 'UTF-16LE')
d.ellis
  • 61
  • 6