Remove empty lines in a text file via grep

Question

FILE:

hello

world

foo

bar

How can I remove all the empty new lines in this FILE?

Output of command:

FILE:

hello
world
foo
bar

Possible duplicate of [Delete empty lines using SED](http://stackoverflow.com/questions/16414410/delete-empty-lines-using-sed) — Ciro Santilli OurBigBook.com, May 16 '16 at 15:34

score 152 · Accepted Answer · edited Nov 14 '13 at 15:31

152

`grep . FILE`

(And if you really want to do it in sed, then: sed -e /^$/d FILE)

(And if you really want to do it in awk, then: awk /./ FILE)

edited Nov 14 '13 at 15:31

Lesmana

25,663
9
82
87

answered Oct 23 '09 at 07:08

DigitalRoss

143,651
25
248
329

1

Sweet! TY for the other commands as well - but looks like grep is my new best friend. – user191960 Oct 23 '09 at 07:15
grep . FILE doesn't work for me. It's probably better to stick with grep for searching file contents, and sed for editing file contents. – Michael Dillon Oct 23 '09 at 08:27
a minor "issue" with grep . FILE is that if a space represents and blank line, then grep will grab that blank line as well. – ghostdog74 Oct 23 '09 at 10:26
7

`sed -ne/./p` works too, and `awk /./` is shorter (action is `{print}` if left unspecified). @ghostdog74: `grep '[^[:space:]]'` then. – ephemient Oct 23 '09 at 16:33
8

For those that don't understand, the `.` is a regular expression that matches any character except for newline. – wisbucky Jan 24 '14 at 05:40
3

`grep . FILE` works with the given example, but not necessarily when the file can have bytes not part of the charset. For instance, with GNU grep 2.20, `printf "\x80\n" | grep .` outputs nothing. – vinc17 Sep 12 '14 at 02:01
The downside of `grep .` compared to the other solutions is that it will highlight all the text in red. The other solutions can preserve the original colors. Compare `unbuffer apt search foo | grep .` to `unbuffer apt search foo | grep -v ^$` – wisbucky Apr 25 '19 at 23:10
`cat FILE | grep -v ^$` works fine. @wisbucky is right – ETech Feb 21 '21 at 20:49

score 47 · Answer 2 · edited May 05 '15 at 16:09

47

Try the following:

grep -v -e '^$'

edited May 05 '15 at 16:09

kenorb

155,785
88
678
743

answered Oct 23 '09 at 08:43

Mr.Ree

8,320
27
30

this has the same effect as grep . FILE in that a space as a blank line will get grabbed. – ghostdog74 Oct 23 '09 at 10:27
1

"grep" looks for any line that matches the pattern. "." matches any character. "grep . FILE" matches any line with at least 1 character. Whereas "grep -v" excludes lines matching the pattern. OP said "remove all the empty new lines". If you want to exclude lines with just spaces, "grep -v '^ *$'". The "*" will match zero or more of the preceding pattern, in this case a space. Though you might prefer to match and exclude other whitespace characters (tabs, form-feeds, etc) too. – Mr.Ree Oct 23 '09 at 23:44
BTW: "^" matches beginning of line. "$" matches the end of line. – Mr.Ree Oct 23 '09 at 23:45
3

This method allowed me to combine multiple excludes more easily than just "grep . FILE". For example, I was looking at a conf file and wanted to exclude all commented lines and all empty lines. So I used "grep -v -e '#' -e '^$' squid.conf". Worked a treat. – Ben K Mar 06 '12 at 13:57
2

this one is a lot faster than the 'grep . FILE'. This is due to the more complex tasks of verifying the regex '.' than excluding as soon as ^$ does not matches. – Édouard Lopez Sep 05 '12 at 11:14
2

`grep -v -e '^$'` always works, which is not the case of `grep .`. For instance, with GNU grep 2.20, `printf "\x80\n" | grep .` outputs nothing, while `printf "\x80\n" | grep -v '^$'` outputs the non-empty line. – vinc17 Sep 12 '14 at 02:06
_"PATTERN is, by default, a basic regular expression (BRE)"._ So `-e` option is not needed. For extended regular expressions `-E` option is needed. – LoMaPh Nov 11 '17 at 22:33

score 13 · Answer 3 · answered Oct 23 '09 at 10:24

13

with awk, just check for number of fields. no need regex

$ more file
hello

world

foo

bar

$ awk 'NF' file
hello
world
foo
bar

answered Oct 23 '09 at 10:24

ghostdog74

327,991
56
259
343

Doesn't need quotes. This trick is found in `awk1line.txt` -- then again, so are most awk tricks :) – ephemient Oct 23 '09 at 16:35
2

its just my good practice to put quotes, since you are running it from shell.. for composite awk statements, you still have to put quotes. so why not cultivate this habit. – ghostdog74 Oct 23 '09 at 23:59
explanation of how this works: https://stackoverflow.com/questions/23544804/how-awk-nf-filename-is-working – wisbucky Apr 25 '19 at 22:14

score 9 · Answer 4 · edited Mar 11 '16 at 19:13

9

Here is a solution that removes all lines that are either blank or contain only space characters:

grep -v '^[[:space:]]*$' foo.txt

edited Mar 11 '16 at 19:13

Adam Katz

14,455
5
68
83

answered Mar 11 '16 at 12:19

Marco Coutinho

420
4
6

kenorb · Answer 5 · 2019-07-22T09:57:41.957

If removing empty lines means lines including any spaces, use:

grep '\S' FILE

For example:

$  printf "line1\n\nline2\n \nline3\n\t\nline4\n" > FILE
$  cat -v FILE
line1

line2

line3

line4
$  grep '\S' FILE
line1
line2
line3
line4
$  grep . FILE
line1
line2

line3

line4

See also:

score 4 · Answer 6 · answered Oct 09 '20 at 04:55

Simplest Answer -----------------------------------------

[root@node1 ~]# cat /etc/sudoers | grep -v -e ^# -e ^$
Defaults   !visiblepw
Defaults    always_set_home
Defaults    match_group_by_gid
Defaults    always_query_group_plugin
Defaults    env_reset
Defaults    env_keep =  "COLORS DISPLAY HOSTNAME HISTSIZE KDEDIR LS_COLORS"
Defaults    env_keep += "MAIL PS1 PS2 QTDIR USERNAME LANG LC_ADDRESS LC_CTYPE"
Defaults    env_keep += "LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT LC_MESSAGES"
Defaults    env_keep += "LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE"
Defaults    env_keep += "LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_CHARSET XAUTHORITY"
Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin
root    ALL=(ALL)       ALL
%wheel  ALL=(ALL)       ALL
[root@node1 ~]#

Prabhat Kumar Singh · Answer 7 · 2014-10-17T11:49:51.740

2

Try this: sed -i '/^[ \t]*$/d' file-name

It will delete all blank lines having any no. of white spaces (spaces or tabs) i.e. (0 or more) in the file.

Note: there is a 'space' followed by '\t' inside the square bracket.

The modifier -i will force to write the updated contents back in the file. Without this flag you can see the empty lines got deleted on the screen but the actual file will not be affected.

edited Oct 17 '14 at 11:49

answered Mar 20 '14 at 14:00

Prabhat Kumar Singh

1,711
15
20

score 1 · Answer 8 · edited Mar 10 '13 at 09:47

1

grep '^..' my_file

example

THIS

IS

THE

FILE

EOF_MYFILE

it gives as output only lines with at least 2 characters.

THIS
IS
THE
FILE
EOF_MYFILE

See also the results with grep '^' my_file outputs

THIS

IS

THE

FILE

EOF_MYFILE

and also with grep '^.' my_file outputs

THIS
IS
THE
FILE
EOF_MYFILE

edited Mar 10 '13 at 09:47

CuberChase

4,458
5
33
52

answered Mar 10 '13 at 06:43

clblue2000

11
1

score 0 · Answer 9 · answered May 05 '15 at 16:10

0

Try ex-way:

ex -s +'v/\S/d' -cwq test.txt

For multiple files (edit in-place):

ex -s +'bufdo!v/\S/d' -cxa *.txt

Without modifying the file (just print on the standard output):

cat test.txt | ex -s +'v/\S/d' +%p +q! /dev/stdin

answered May 05 '15 at 16:10

kenorb

155,785
88
678
743

Chris Koknat · Answer 10 · 2015-10-13T23:42:37.397

0

Perl might be overkill, but it works just as well.

Removes all lines which are completely blank:

perl -ne 'print if /./' file

Removes all lines which are completely blank, or only contain whitespace:

perl -ne 'print if ! /^\s*$/' file

Variation which edits the original and makes a .bak file:

perl -i.bak -ne 'print if ! /^\s*$/' file

edited Oct 13 '15 at 23:42

answered Sep 10 '15 at 18:15

Chris Koknat

3,305
2
29
30

score 0 · Answer 11 · answered Aug 22 '20 at 08:03

If you want to know what the total lines of code is in your Xcode project and you are not interested in listing the count for each swift file then this will give you the answer. It removes lines with no code at all and removes lines that are prefixed with the comment //

Run it at the root level of your Xcode project.

find . \( -iname \*.swift \) -exec grep -v '^[[:space:]]*$' \+ | grep -v -e '//' | wc -l

If you have comment blocks in your code beginning with /* and ending with */ such as:

/*
 This is an comment block 
*/

then these will get included in the count. (Too hard).

Remove empty lines in a text file via grep

11 Answers11

`grep . FILE`

Linked

Related