Why do I need vim in binary mode for 'noeol' to work?

Question

This question is a follow up to the Work Around I use for "saving files in vim without having the newline at end of file forcibly added" annoyance.

Basically I can't use set noeol in my .vimrc because it does Nothing!

It does what it's supposed to do though if I edit the file in binary mode. (vim -b file instead of vim file)

Why is that?

Anyway to have a simple preference in .vimrc to NOT add newlines at every single file I edit?

Also, what kind of issues will I encounter if I start to edit every file in binary mode? So far I haven't seen any difference.

score 118 · Accepted Answer · edited Apr 17 '20 at 18:58

118

What Vim "adds" to the end of the last line in your file is the "newline" character, which should not to be confused with a "new line".

The "newline" character or more accurately "end of line" character (<EOL>) means "whatever comes after this point must be considered to be on another line". With this interpretation — <EOL> is a line terminator — the last line of the file is effectively the last one with an <EOL>.

The problem is that most editors and IDEs have a different interpretation — <EOL> is a line separator — and, logically, default to not add an <EOL> at the end of the last line of a new file and, when they encounter an <EOL>, add a superfluous "new line" after the real last line.

In short, Vim doesn't add a "new line": other editors interpret (wrongly) its "newline" as a "new line".

But you can get around that issue by doing the following: before you write your file, do :set binary noeol if you want it to stay "<EOL>-free".

However, :h 'binary' has a lot to say about the perils of :set binary so I'd say that turning it "on" all the time sounds like a bad idea.

To illustrate the different behaviors, this is what happens when you try to concatenate two files with <EOL>:

$ cat file1    $ cat file2         $ cat file1 file2

lorem ipsum    Le tramway jaune    lorem ipsum
dolor sit      avance lentement    dolor sit
amet           dans le             amet
                                   Le tramway jaune
                                   avance lentement 
                                   dans le

and this is what happens when you try to concatenate two files without <EOL>:

$ cat file1    $ cat file2         $ cat file1 file2

lorem ipsum    Le tramway jaune    lorem ipsum
dolor sit      avance lentement    dolor sit
amet           dans le             ametLe tramway jaune
                                   avance lentement 
                                   dans le

The first behavior is somehow the expected behavior and the reason why Vim and many (if not most) UNIX-y programs default to the terminator interpretation and to adding an <EOL> character at the end of the last line.

The picture below shows a simple file with <EOL> created with nano (it would be the same with Vim) and opened in Eclipse, TextMate, Sublime Text, Vim, Xcode and TextEdit.

<EOL>

(edit) There is no line 4 in this file and the only editor of the bunch that displays the file correctly is Vim. A line number column's only purpose is to provide information on the buffer. Showing 4 lines where there are only 3 is a gross mistake. (endedit)

This picture shows another simple file without <EOL> created with Sublime Text and opened in the same editors/IDEs.

No <EOL>

edited Apr 17 '20 at 18:58

Boris Verkhovskiy

14,854
11
100
103

answered Apr 25 '13 at 20:39

romainl

186,200
21
280
313

12

and upvote for the excellent explanation on EOL. but the issue is to have control over that. I want to decide, not leave that to vim. – gcb Apr 28 '13 at 03:46
2

I think that you only need to write a small function that temporarily `set binary noeol` and bind it to the `BufWritePre` event. – romainl Apr 28 '13 at 06:21
2

Even better than `set binary noeol` would be `setl binary noeol`. – bdesham Nov 25 '13 at 21:24
2

Excellent explanation and illustration. Wish I could vimlove+2 for the column text formatting in the `$ cat file1 ` section. By the way, Atom is in the 'Shows line # 4' club. – Will Jun 16 '16 at 17:32
1

While this may be a technically well written _explanation_ I find the _justification_ for the persistence in appending an `` rather weak. I spend most of my time changing the content within files, not concatenating files together...so the interpretation of last line and behavior of concatenating files is irrelevant to me (this arises maybe, 1 time per year?). On the other hand I am *constantly* changing file content, I have never had any issues with the religious opinions of editors, but extraneous character diffs certainly generate unhelpful VCS noise, so I consider this purely a wart. – aaron Nov 10 '16 at 17:39
That unhelpful VCS noise is a direct consequence of the differences between editors explained in my answer so yeah, it's probably more relevant to you than you think. – romainl Nov 10 '16 at 20:18
2

I don't agree with your interpretation here, @romainl. A typical text editor renders the newline character as a line break -- moving to the next line. Vim doesn't render it at all. So when concatenating files, you get a behavior that matches the typical text editor and differs from Vim. – Nathan Hinchey Jan 31 '18 at 22:32
1

@NathanHinchey it's not my interpretation, it has been the prevalent interpretation among text editor and CLI tool authors for decades. Also, Vim renders the final newline [fine](https://i.imgur.com/PHOobJE.png) if you ask it to (`:help 'list'`). – romainl Jan 31 '18 at 23:02
1

Okay, having done more reading that starts to make sense. And if we have a file `noeol.txt` which contains no EOL characters, then `wc -l noeol.txt` returns 0. So would it be accurate to say that text (not in a binary file) that does not end with an EOL is malformed? – Nathan Hinchey Feb 01 '18 at 15:28
3

@NathanHinchey I would be tempted to consider it either as a stream of some kind or as a malformed file depending on the context. – romainl Feb 01 '18 at 18:01
3

Are there any questions asking the opposite: why do some editors show a blank line at the end of files? – Nathan Hinchey Feb 01 '18 at 20:24
3

I'm solidly in the camp of EOL being required as line terminator, even at EOF (frequent Vim user). However, to play devil's advocate: if the other editors didn't display a blank line after the final EOL, how would a user position the cursor there in order to add a line after that EOL? This is especially true for TextEdit. For TextMate, if the file only had 3 lines, maybe a gray square could appear under the "3" until the user starts typing, at which point it becomes a "4" and the 4th EOL is added automatically. – Kelvin Mar 28 '19 at 20:34
1

@Kelvin, the user would move the cursor on the last character of the last line and press `` just like he would add a line before any existing line. – romainl Mar 30 '19 at 16:29
1

I disagree with your **premise** that other editors interpret wrongly. I've run with that problem with kubernetes by taking secret values from file. The values are **not** interpreted, which means, that surplus was added to the passwords... – 9ilsdx 9rvj 0lo Oct 07 '19 at 07:09
in those other editors, there is no 4th line either. the 4th row is empty, and contains nothing. the editor displaying a 4th linenumber is no different than vim's dumbfuck 'listchars' still displaying an eol char in a file that has no newline at end of file. i'm starting to lean against vim's choice on this issue – mikew Nov 09 '22 at 01:33
@romainl you wrote: "Also, Vim renders the final newline fine if you ask it to". No, it does not. Turn on 'list' and you'll see the trailing $ char at end of file, even when there is no newline at end of file. Broken feature. – mikew Nov 09 '22 at 01:36
Broken or misunderstood? My comment may have been misleading, and I'm sorry if that's the case, but `:help 'list'` is not meant to provide information about the *raw characters* in the file. It is meant to help the user visualize a few things about the *text* in the buffer, like trailing whitespace, differentiate tabs and spaces, etc. `[noeol]` in the status line is how Vim tells you that there is no newline at the end of file. Whether that is enough or good looking is debatable but the information is there. If you want more, make a PR or write a plugin. – romainl Nov 09 '22 at 06:45
@romainl The problem is that `[noeol]` not only does not persist in the statusline, but even if it did, Vim does not keep track of the eol status. It only checks it once upon file open. Simply open up a noeol file, use `:set eol?` , edit the file, save it, Vim auto adds the eol, then use `:set eol?` again and Vim will still incorrectly report `noeol` – mikew Nov 09 '22 at 17:47

mrajner · Answer 2 · 2018-08-15T14:43:37.813

25

from version 7.4.785 vim has fixendofline setting. You can avoid binary (which has some side effects) and simply set

set noendofline
set nofixendofline

edited Aug 15 '18 at 14:43

answered Sep 21 '16 at 22:16

mrajner

392
4
9

score 5 · Answer 3 · answered Apr 25 '13 at 19:39

According to the vimdoc noeol does nothing unless binary mode is on.

            *'endofline'* *'eol'* *'noendofline'* *'noeol'*
'endofline' 'eol'   boolean (default on)
            local to buffer
            {not in Vi}
    When writing a file and this option is off and the 'binary' option
    is on, no <EOL> will be written for the last line in the file.  This
    option is automatically set when starting to edit a new file, unless
    the file does not have an <EOL> for the last line in the file, in
    which case it is reset.  Normally you don't have to set or reset this
    voption.  When 'binary' is off the value is not used when writing the
    file.  When 'binary' is on it is used to remember the presence of a
    <EOL> for the last line in the file, so that when you write the file
    the situation from the original file can be kept.  But you can change
    it if you want to.

The only actual answer. Although @mrajner answer is more useful :) — akostadinov, Aug 29 '20 at 12:32

score 3 · Answer 4 · answered Apr 26 '13 at 12:11

3

have a simple preference in .vimrc to NOT add newlines at every single file i edit

You can use my PreserveNoEOL plugin for that. With this simple setting, you're done; alternatively, you can also influence this per buffer:

:let g:PreserveNoEOL = 1

answered Apr 26 '13 at 12:11

Ingo Karkat

167,457
16
250
324

interesting. but your script is not something that i can easily add to my vimrc :) and not very fan of the way it works. it's still does not gives me practical control. i will probably try to make it simpler and in a way that it simply add a new line if the file has a EOL and only saves it if the buffer is saved with a newline. Your code is neat and full of comments, will be a nice starting point! – gcb Apr 28 '13 at 03:49
Well, you just install it like any other plugin. I don't fully understand your criticism; if you `let g:PreserveNoEOL = 1`, it should "just work", i.e. preserve existing files with no trailing EOL, and leave all other (regular) files alone. Anyway, it's open source, so you're free to (re-)use whatever you see fit. I'm also open to general feedback and suggestions, via email. – Ingo Karkat Apr 28 '13 at 08:30
1

it's all fine when you work from one machine. but i'm working from 2 to 12 different machines every day at work. Even copying my `bashrc` and `vimrc` is already a pain :/ i simply had to learn to live without plugins. It's not a criticism to your work. just saying that i can't have the luxury, unfortunately. – gcb Jul 10 '13 at 16:59

Why do I need vim in binary mode for 'noeol' to work?

4 Answers4

Linked