3

I am a new Ubuntu user dealing with a very large file with a few non-utf8 characters that can be safely skipped. I found another stack overflow question How to remove non UTF-8 characters from text file that gave a way to remove those characters, using the command 

iconv -f utf-8 -t utf-8 -c file.txt

however with the size of my file, this outputs every line, which takes too much time. I'm not too familiar with Ubuntu commands so if anyone could guide me on modifying that command to suppress outputs, I would appreciate it

Community
  • 1
  • 1
Nimish Todi
  • 137
  • 1
  • 11

2 Answers2

5

Output the conversion to a new file using shell redirection:

iconv -f utf-8 -t utf-8 -c file.txt > new-file.txt

Then check the end of new file:

tail new-file.txt

Check the top:

head new-file.txt
Alastair McCormack
  • 26,573
  • 8
  • 77
  • 100
2

You can redirect the output to a new file, instead of printing every line on the terminal:

iconv -f utf-8 -t utf-8 -c file.txt > output.txt
Francisca Concha-Ramírez
  • 912
  • 2
  • 10
  • 25