38

I want to use iconv to convert files on my Mac. The goal is to go from "Windows ANSI" to "whatever Windows Notepad saves, if you tell it to use UFT8".

This is what I want:

$ file names.csv 
names.csv: UTF-8 Unicode (with BOM) text, with CRLF line terminators

This is what I use:

$ iconv -f CP1252 -t UTF-8  names.csv > names.utf8.csv

This is what I get (not what I want):

$ file names.utf8.csv 
names.utf8.csv: UTF-8 Unicode text, with CRLF line terminators

How do I get the BOM?

jsageryd
  • 4,234
  • 21
  • 34
user531912
  • 383
  • 1
  • 3
  • 5

4 Answers4

58

You can add it manually by first echoing the bytes into the file:

echo -ne '\xEF\xBB\xBF' > names.utf8.csv

and then concatenating your required information at the end:

iconv -f CP1252 -t UTF-8  names.csv >> names.utf8.csv

Note the >> rather than >.

borrible
  • 17,120
  • 7
  • 53
  • 75
  • Any idea why iconv from GnuWin is creating a UCS2 LE BOM encoded file (according to Notepad++ anyway) when I use `-t UTF-8` ? – Fuhrmanator Sep 22 '17 at 18:20
  • See https://stackoverflow.com/questions/3604753/iconv-is-converting-to-utf-16-instead-of-utf-8-when-invoked-from-powershell – Fuhrmanator Sep 22 '17 at 19:03
1

Note that "Windows ANSI" may not be CP1252 - that is configured by users.

Nemanja Trifunovic
  • 24,346
  • 3
  • 50
  • 88
-3

The BOM is not necessary for UTF-8. And Windows Notepad can save UTF-8 with or without BOM.

Mike11
  • 37
  • 2
-7

I needed the opossite. (covert german text from UTF-8 to ANSI)

So command I used:
1. iconv -l (check available formats)
2. iconv -f UTF8 -t MS-ANSI de.txt > output.txt

and now if I open output.txt it is already in ANSI. Job done.

Maris
  • 13
  • 3