14

I have a bunch of text files that are encoded in ISO-8851-2 (have some polish characters). Is there a command line tool for linux/mac that I could run from a shell script to convert this to a saner utf-8?

Jon Seigel
  • 12,251
  • 8
  • 58
  • 92
Marcin
  • 7,874
  • 7
  • 45
  • 49
  • 4
    Most likely ISO-885**9**-2. ISO 8851 speaks about butter. – Melebius Feb 08 '16 at 10:37
  • Possible duplicate of [Best way to convert text files between character sets?](https://stackoverflow.com/questions/64860/best-way-to-convert-text-files-between-character-sets) – MultiplyByZer0 Mar 23 '19 at 20:03

3 Answers3

29

Use iconv, for example like this:

iconv -f LATIN1 -t UTF-8 input.txt > output.txt

Some more information:

  • You may want to specify UTF-8//TRANSLIT instead of plain UTF-8. To quote the manpage:

    If the string //TRANSLIT is appended to to-encoding, characters being converted are transliterated when needed and possible. This means that when a character cannot be represented in the target character set, it can be approximated through one or several similar looking characters. Characters that are outside of the target character set and cannot be transliterated are replaced with a question mark (?) in the output.

  • For a full list of encoding codes accepted by iconv, execute iconv -l.

  • The example above makes use of shell redirection. Make sure you are not using a shell that mangles encodings on redirection – that is, do not use PowerShell for this.
MultiplyByZer0
  • 6,302
  • 3
  • 32
  • 48
lhf
  • 70,581
  • 9
  • 108
  • 149
10
recode latin2..utf8 myfile.txt

This will overwrite myfile.txt with the new version. You can also use recode without a filename as a pipe.

legoscia
  • 39,593
  • 22
  • 116
  • 167
  • 1
    Way more efficient than accepted answer, because iconv won't replace the same file, even using -o or output redirects. – Julien Nov 15 '10 at 11:42
3

GNU 'libiconv' should be able to do the job.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278