1

Below is the data in my file:

one_1.10=to - Standard (£1.10)
ITV_1.10=to - Standard (£1.10)

I want to remove  in my file in unix?

i tried below methods. Opened the file in vi mode and gave below commands

:set nobomb
:wq

sed -i '1 s/^\xef\xbb\xbf//' <fileName>

But the BOM character is not getting removed. Could someone please help?

michael
  • 9,161
  • 2
  • 52
  • 49
user2531569
  • 609
  • 4
  • 18
  • 36
  • Does `sed -i 's/Â//g' file` work to you? – fedorqui Sep 20 '13 at 12:40
  • What you're trying to remove is definitely not a [BOM](http://en.wikipedia.org/wiki/Byte_order_mark). – devnull Sep 20 '13 at 12:45
  • Hi fedorqui. Its not working. its showing like pattern not found, though the pattern is there. – user2531569 Sep 20 '13 at 12:50
  • Hi devnull. I thought its a BOM charac.. Anyway i m just trying to remove that character. – user2531569 Sep 20 '13 at 12:51
  • I found another way to replace all non-ascii characters in all files inside the current directory : find . -type f | xargs perl -pi.bak -e 's,[^[:ascii:]],,g' afterwards you will have to find and remove all the '.bak' files: find . -type f -a -name \*.bak | xargs rm but this is also replacing £ with empty string. does anyone has any idea like how to exclude £ by using above commands? – user2531569 Sep 23 '13 at 06:56
  • Fyi: original title was "How to remove BOM character  character in a file in unix?", but that's not what this question is about. If that's what you're looking for, see https://stackoverflow.com/questions/1068650/using-awk-to-remove-the-byte-order-mark – michael Jul 29 '17 at 18:46
  • It's not clear what is in the file. Consider sharing hexdump of the data file (hexdump -n32 -c, or similar) – dash-o Nov 21 '19 at 06:44

1 Answers1

0

Try:

cat <filename> | sed 's/\xc2//g'

You had the right idea, but the wrong character (based on your example).

Evil Genius
  • 429
  • 5
  • 13
  • I don't think the text you are trying to work with is getting represented properly above. When I paste your example text into a file on my system, the 'Â' looks like a 0xC2 character when viewed in a hex editor. Try viewing your original file in hex and replace the \xc2 with whatever character it really is. For viewing hex: emacs can do this (open the file then press Esc., x, type hexl-mode and press enter). Or run 'od -h '. – Evil Genius Sep 20 '13 at 13:13