1

I'm trying to use a sed one-liner to remove trailing newlines using one of the handy sed one-liners:

sed --in-place -e :a -e '/^\n*$/{$d;N;ba}'

It works for UNIX-style newlines, but doesn't for DOS-style newlines. Despite the fact that I couldn't exactly get my head around sed scripting syntax and matching rules, I tried this:

sed --binary --in-place -e :a -e '/^\(\r\?\n\)*$/{$d;N;ba}'

but it didn't help, and I don't really know why.

I don't really want to use Perl, etc., I want to stick to more "core" utilities. So, are there any sed gurus around? I would really appreciate your help. I'm using Cygwin's distribution of sed.

Thank you.

Specifically, I want this file (printed out using cat -A):

1^M$
^M$
^M$

to become this:

1$
Egor Tensin
  • 576
  • 4
  • 17
  • dos2unix the file and then run your sed thing. – pvg Dec 13 '15 at 18:04
  • @pvg I want to create like an alias in my `.bashrc`, so would really like if it was a single command, cause piping `dos2unix` and `sed` is trickier than running `sed` alone. – Egor Tensin Dec 13 '15 at 18:09
  • Shouldn't really be a problem. Or just make it a bash function, easier to read and maintain. http://stackoverflow.com/questions/756756/multiple-commands-in-an-alias-for-bash – pvg Dec 13 '15 at 18:11
  • @pvg Yeah, but then each file would require a separate `sed` (`dos2unix`, etc.) call... I know I can just make a function, but I'm just wondering if I can do it simpler. – Egor Tensin Dec 13 '15 at 18:14
  • If you only want to remove the newlines from the end of the file but not from the middle of it, that's not a good sample input/output. – Ed Morton Dec 13 '15 at 19:15

3 Answers3

2

You're trying to use the wrong tool for the job. sed is for simple subsitutions on individual lines, that is all. For anything even sightly more interesting you should be using awk, e.g. one approach with GNU awk would be:

$ cat -v file
1^M
^M
^M
$ awk -v RS='^$' -v ORS= '{sub(/(\r?\n)+$/,"\n")}1' file | cat -v
1

with other awks:

$ awk '{rec = rec $0 RS} END{sub(/(\r?\n)+$/,"\n",rec); printf "%s",rec}' file | cat -v
1

The above will work whether the line endings are DOS or UNIX.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • 1
    Hey, your answer works for me! My command messed up with the output newline characters.. At least on Linux.. – hek2mgl Dec 13 '15 at 19:35
0

Could you use tr like this?

tr -d '\r\n' < file1 > tempfile && mv tempfile file1

This translates file1 and removes \r\n.

zedfoxus
  • 35,121
  • 5
  • 64
  • 63
  • I could, but I really wanted to do this using a single program (like `sed`), cause I wanted to `alias` the whole thing in my `.bashrc`. Creating a function would require multiple `sed` calls for a bunch of files, etc., so would really appreciate a single program call. – Egor Tensin Dec 13 '15 at 18:12
0

Yeah, this worked:

sed --binary --in-place -e :a -e '/^\(\r\n\)*\r$/{$d;N;ba}'

Would prefer to delete the question really, but it's too late.

Egor Tensin
  • 576
  • 4
  • 17