use sed to merge lines and add comma

Question

I found several related questions, but none of them fits what I need, and since I am a real beginner, I can't figure it out.

I have a text file with entries like this, separated by a blank line:

example entry &with/ special characters
next line (any characters)

next %*entry
more words

I would like the output merge the lines, put a comma between, and delete empty lines. I.e., the example should look like this:

example entry &with/ special characters, next line (any characters)
next %*entry, more words

I would prefer sed, because I know it a little bit, but am also happy about any other solution on the linux command line.

Should commas in the data be disambiguated in the output somehow? — tripleee, May 08 '14 at 08:30
So is an empty line an entry separator, or do you always have exactly two lines which should be joined together? — tripleee, May 08 '14 at 08:31

Mark Setchell · Accepted Answer · 2014-05-08T08:46:19.230

8

Improved per Kent's elegant suggestion:

awk 'BEGIN{RS="";FS="\n";OFS=","}{$1=$1}7' file

which allows any number of lines per block, rather than the 2 rigid lines per block I had. Thank you, Kent. Note: The 7 is Kent's trademark... any non-zero expression will cause awk to print the entire record, and he likes 7.

You can do this with awk:

awk 'BEGIN{RS="";FS="\n";OFS=","}{print $1,$2}' file

That sets the record separator to blank lines, the field separator to newlines and the output field separator to a comma.

Output:

example entry &with/ special characters,next line (any characters)
next %*entry,more words

edited May 08 '14 at 08:46

answered May 08 '14 at 08:37

Mark Setchell

191,897
31
273
432

2

+1 however after the `BEGIN` block, I would do `... '{$1=$1}7' file` for the cases that in a "block" there are more than two lines. – Kent May 08 '14 at 08:41
op wants `space` after comma. – Avinash Raj May 08 '14 at 09:17
Your code should be `awk 'BEGIN{RS="";FS="\n";OFS=", "}{print $1,$2}'` – Avinash Raj May 08 '14 at 09:22

score 2 · Answer 2 · edited May 23 '17 at 12:20

2

Simple sed command,

sed ':a;N;$!ba;s/\n/, /g;s/, , /\n/g' file

:a;N;$!ba;s/\n/, /g -> According to this answer, this code replaces all the new lines with ,(comma and space).

So After running only the first command, the output would be

example entry &with/ special characters, next line (any characters), , next %*entry, more words

s/, , /\n/g - > Replacing , , with new line in the above output will give you the desired result.

example entry &with/ special characters, next line (any characters)
next %*entry, more words

edited May 23 '17 at 12:20

Community

1
1

answered May 08 '14 at 09:14

Avinash Raj

172,303
28
230
274

you have to treat the case where there is `, , ` into the text or`, ` at the end of one line. if not occuring, your code is fast and simple – NeronLeVelu May 08 '14 at 12:32

score 2 · Answer 3 · answered May 08 '14 at 12:14

This might work for you (GNU sed):

sed ':a;$!N;/.\n./s/\n/, /;ta;/^[^\n]/P;D' file

Append the next line to the current line and if there are characters either side of the newline substitute the newline with a comma and a space and then repeat. Eventually an empty line or the end-of-file will be reached, then only print the next line if it is not empty.

Another version but a little more sofisticated (allowing for white space in the empty line) would be:

sed ':a;$!N;/^\s*$/M!s/\n/, /;ta;/\`\s*$/M!P;D' file

score 0 · Answer 4 · answered May 08 '14 at 08:44

sed -n '1h;1!H
$ {x
   s/\([^[:cntrl:]]\)\n\([^[:cntrl:]]\)/\1, \2/g
   s/\(\n\)\n\{1,\}/\1/g
   p
   }' YourFile

change all after loading file in buffer. Could be done "on the fly" while reading the file and based on empty line or not.

use -e on GNU sed

use sed to merge lines and add comma

4 Answers4