What does '^@' mean in vim?

Question

When I cat a file in bash I get the following:

$ cat /tmp/file 
microsoft

When I view the same file in vim I get the following:

^@m^@i^@c^@r^@o^@s^@o^@f^@t^@

How can I identify and remove these "non-printable" characters. What does '^@' mean in vim??

(Just a piece of background information: the file was created by base 64 decoding and cutting from the pssh header of an mpd file for Microsoft Playready)

I should not that when I type 'ga' in normal mode it does reveal the character: <^@> 0, Hex 00, Octal 000 — murungu, Nov 23 '17 at 13:59

score 7 · Accepted Answer · answered Nov 23 '17 at 14:31

What you see is Vim's visual representation of unprintable characters. It is explained at :help 'isprint':

Non-printable characters are displayed with two characters:
    0 -  31   "^@" - "^_"
   32 - 126   always single characters
     127      "^?"
  128 - 159   "~@" - "~_"
  160 - 254   "| " - "|~"
     255      "~?"

Therefore, ^@ stands for a null byte = 0x00. These (and other non-printable characters) can come from various sources, but in your case it's an ...

encoding issue

If you clearly observe your output in Vim, every second byte is a null byte; in between are the expected characters. This is a clear indication that the file uses a multibyte encoding (utf-16, big endian, no byte order mark to be precise), and Vim did not properly detect that, and instead opened the file as latin1 or so (whereas things worked out properly in the terminal).

To fix this, you can either explicitly specify the encoding:

:edit ++enc=utf-16 /tmp/file

Or tweak the 'fileencodings' option, so that Vim can automatically detect this. However, be aware that ambiguities (as in your case) make this prone to fail:

For an empty file or a file with only ASCII characters most encodings will work and the first entry of 'fileencodings' will be used (except "ucs-bom", which requires the BOM to be present).

That's why a byte order mark (BOM) is recommended for 16-bit encodings; but that assumes that you have control over the output encoding.

score 4 · Answer 2 · edited Nov 23 '17 at 21:53

4

^@ is Vim's representation of a null byte. The ^ indicates a non-printable control character, with the following ASCII character indicating which control character it is.

^@ == 0 (NUL)
^A == 1
^B == 2
...
^H == 8
^K == 11
...
^Z == 26
^[ == 27
^\ == 28
^] == 29
^^ == 30
^_ == 31
^? == 127

9 and 10 aren't escaped because they are Tab and Line Feed respectively.

32 to 126 are printable ASCII characters (starting with Space).

edited Nov 23 '17 at 21:53

ikegami

367,544
15
269
518

answered Nov 23 '17 at 14:04

chepner

497,756
71
530
681

thanks for your answer. It is direct and to the point. The answer I accepted gave me a bit more background information which is why I accepted it. However I have up-voted your answer as well. – murungu Nov 23 '17 at 14:47
No problem; Ingo clearly answers your question in a more useful manner, – chepner Nov 23 '17 at 15:03

What does '^@' mean in vim?

2 Answers2

encoding issue