Replace two newlines with a single one and more than two with two newlines

Question

I have a file which looks like this:

Line 1

Line 2

Line 3



Line 4

Line 5



Line 6

How can I make it look like this:

Line 1
Line 2
Line 3

Line 4
Line 5

Line 6

I.e. replace two consecutive newlines with a single one and more than two with two newlines?

Answer 1

If we look at the sequences of line feeds, this is what we want:

\n       -> \n    (No change)
\n\n     -> \n
\n\n\n+  -> \n\n

The simple solution involves loading the entire file into memory.

perl -0777pe's/\n\n?\K\n+//g'

If you want to avoid that, you can use the following:

perl -ne'
   chomp;
   $b = length ? 0 : $b+1;
   CORE::say if $b==0 || $b==2;
'

$b stands for "blank", and contains the number of blank lines encountered in a row.

See Specifying file to process to Perl one-liner.

Answer 2

In Perl: To match consecutive newlines using a regex, you cannot read in line-by-line mode. Which is why we slurp the file into a single string.

my $str = do { local $/; <DATA> };   # slurp the file into a single string
$str =~ s/\n\n?\K\n+//g;             
print $str;

The substitution regex matches a single newline \n, followed by an optional newline \n?, which it keeps \K, followed by 1 or more newlines \n+, which it removes. Since all quantifiers are greedy, this will allow the ? to preserve the case of two newlines when there are 3 or more.

Case     \n\n?\K\n+    explanation                   result
\n        1 x    x     no match, no substitution     no change
\n\n      1 0    1     match, skip, match 1 time     \n remove \n
\n\n\n+   1 1    1+    match, match, match 1+ times  \n\n remove \n+

Or if you like it as a one-liner:

perl -0777 -pe's/\n\n?\K\n+//g' file

Add -i option to in-place edit a file when you are satisfied the changes work as expected. -i.bak to save backup.

Answer 3

Well, I figured it out myself:

perl -0777 -i -pe 's/\n\n/\n/g' file

Answer 4

Solution using awk:

awk 'BEGIN {minus2 = "a"; minus1 = "a";}{if($0==""){if(minus1=="" && minus2!=""){print $0}}else{print $0}; minus2 = minus1; minus1 = $0}' yourfile.txt

Explanation: to decide if to print current line I need to know content of two previous lines - I keep it as minus2 and minus1. In BEGIN I set them (any value can be used, which !=""). Above might be presented in pseudo-code as follow for every line do:

if line is empty:
    if previous line is empty and previous previous non-empty:
        print line
    else:
        do nothing
else:
    print line

Then I update minus1 and minus2 so it would have correct values for next line. Simply speaking I did print solely 2nd empty line from every group of empty lines.