0

Note:

This question is different from the marked question because This question deals with write of the data in a CSV file rather than only dumping.

Question

I have a data structure which I want to dump into a CSV, but that data contains, Unicode Chars in Japanese. When I dump the data in CSV using Python2.7 it doesn't render at all.

Data:

u'\u6700\u7d42\u30ec\u30dd\u30fc\u30c8(\u65e5\u672c\u8a9e) /Final Report in Japanese'

Code

data = u'\u6700\u7d42\u30ec\u30dd\u30fc\u30c8(\u65e5\u672c\u8a9e) /Final Report in Japanese'
output_buffer.write(codecs.BOM_UTF8)
csvwriter = csv.writer(output_buffer)
encoded_data =six.text_type(data).encode('utf-8')
rows_to_write = [['title', encoded_data]]
csvwriter.writerows(rows_to_write)

The above code writes the data correctly and when I open it inside the editor (MS Excel, Google Sheets, and PyCharm), it renders Japanese correctly. The issue is when

Issue

..
..
encoded_data =six.text_type(data).encode('utf-8')

rows_to_write = [['title', json.dumps('data': encoded_data)]]

csvwriter.writerows(rows_to_write)

This is where when I see the data within the editor(MS Excel, Google Sheets, and PyCharm), it does not render correctly and just see the codes rather than rendered Japanese.

Any suggestions?

A.J.
  • 8,557
  • 11
  • 61
  • 89
  • Take care with editor, it should have an encoding selector to "render correctly", check if you find it or post your editor – Wonka Aug 02 '19 at 06:57
  • Tried in the following editors. (MS Excel, Google Sheets, and PyCharm) – A.J. Aug 02 '19 at 07:01
  • Are you sure your first code is working correctly ? `rows_to_write` should be list of list e.g. : `[['title', encoded_data]]`, `[['title', json.dumps('data': encoded_data)]]` – ymonad Aug 02 '19 at 07:15
  • thanks for pointing it out, i just used sample data from and I am sure the data inside the app is working correctly, I am giong to update the question. – A.J. Aug 02 '19 at 07:18
  • 2
    Re "not a duplicate": To point this out more clearly: you're getting `\u....` *from `json.dumps`*, not from anything to do with CSV. You need to tell `json.dumps` to encode your data as desired! – deceze Aug 02 '19 at 07:31

0 Answers0