Read a csv file containing special characters(different spoken language)

Question

I'm writing a csv file in Hindi(language), when I parse it I get different results.

For example, I make below csv file.

1234444070;आज आप कैसे हैं???

When I read the same file using open(csv_aws_url).read. I get:

"1234444070;\xE0\xA4\x86\xE0\xA4\x9C \xE0\xA4\x86\xE0\xA4\xAA \xE0\xA4\x95\xE0\xA5\x88\xE0\xA4\xB8\xE0\xA5\x87 \xE0\xA4\xB9\xE0\xA5\x88\xE0\xA4\x82???\r\n"

Can this happen that I read the same contents while parsing?

Try putting `puts` before your `open`. It looks like you're just seeing the `String#inspect` output, which is just a different visual representation of the same data. — Jordan Running, Jul 08 '16 at 13:26

score 2 · Answer 1 · edited May 23 '17 at 11:51

2

Try open(csv_aws_url, encoding: "utf-8").read

The file is most likely being saved with a different encoding.

Ruby read CSV file as UTF-8 and/or convert ASCII-8Bit encoding to UTF-8 should be helpful.

edited May 23 '17 at 11:51

Community

1
1

answered Jul 08 '16 at 11:58

Prakash Murthy

12,923
3
46
74

`open(csv_aws_url).read.force_encoding('utf-8')` did the trick! – bill_cosby Jul 15 '16 at 03:37
This did the trick for me, thank you. – Luke DeWitt Jan 27 '22 at 00:37

score 1 · Accepted Answer · answered Jul 15 '16 at 03:38

1

open(csv_aws_url).read.force_encoding('utf-8')

answered Jul 15 '16 at 03:38

bill_cosby

145
2
9

Read a csv file containing special characters(different spoken language)

2 Answers2