sed join lines together

Question

what would be the sed (or other tool) command to join lines together in a file that do not end w/ the character '0'?

I'll have lines like this

412|n|Leader Building Material||||||||||d|d|20||0

which need to be left alone, and then I'll have lines like this for example (which is 3 lines, but only one ends w/ 0)

107|n|Knot Tying Tools|||||Knot Tying Tools

|||||d|d|0||0

which need to be joined/combined into one line

107|n|Knot Tying Tools|||||Knot Tying Tools|||||d|d|0||0

ninjalj · Accepted Answer · 2011-10-21T19:17:17.297

57

 sed ':a;/0$/{N;s/\n//;ba}'

In a loop (branch ba to label :a), if the current line ends in 0 (/0$/) append next line (N) and remove inner newline (s/\n//).

awk:

awk '{while(/0$/) { getline a; $0=$0 a; sub(/\n/,_) }; print}'

Perl:

perl -pe '$_.=<>,s/\n// while /0$/'

bash:

while read line; do 
    if [ ${line: -1:1} != "0" ] ; then 
        echo $line
    else echo -n $line
fi
done

edited Oct 21 '11 at 19:17

answered Oct 21 '11 at 18:43

ninjalj

42,493
9
106
148

1

question says join if line does `NOT` end with '0'. Given sed does the opposite. – Kashyap Feb 07 '14 at 01:32
Here is the one that solves the problem OP asked for http://stackoverflow.com/questions/20094997/merge-lines-between-two-patterns-using-sed – Kashyap Feb 07 '14 at 02:13
1

The bash loop, even though it looks uglier, it's actually quite a bit faster, more flexible and easier to understand and debug. I was able to adapt it to join lines where the criteria was: join each line starting with "https": with all the following lines until the next one that starts with "https:". – miguev Feb 28 '18 at 21:48
On macOS, the branching didn't work for me, but removing it did: `sed '/my pattern/{N;s/\n//;}'` – Henry Blyth Apr 08 '21 at 16:28

score 10 · Answer 2 · answered Oct 21 '11 at 21:15

10

awk could be short too:

awk '!/0$/{printf $0}/0$/'

test:

kent$  cat t
#aasdfasdf
#asbbb0
#asf
#asdf0
#xxxxxx
#bar

kent$  awk '!/0$/{printf $0}/0$/' t
#aasdfasdf#asbbb0
#asf#asdf0
#xxxxxx#bar

answered Oct 21 '11 at 21:15

Kent

189,393
32
233
301

3

Hi! Could you add a short description on how this works please? :) – lima.sierra Jun 02 '16 at 15:27
it is very straightforward which part you have problem to understand? – Kent Jun 02 '16 at 15:28
Well: (1) What command is that, with four slashes? (2) How does that last "0$" part work? (3) Generally, how to search for documentation for this command – lima.sierra Jun 02 '16 at 15:52
1

Having played around with this for almost an hour now, can you confirm the following explanation: For every line which does not have 0 at the end, print that line (without newline). Then, another regex begins: For every line which _has_ a 0 at the end, don't touch it (which result in "print it normally, including it's newline") – lima.sierra Jun 02 '16 at 16:22

score 4 · Answer 3 · edited May 23 '17 at 12:10

The rating of this answer is surprising ;s (this surprised wink emoticon pun on sed substitution is intentional) given the OP specifications: sed join lines together.

This submission's last comment

"if that's the case check what @ninjalj submitted"

also suggests checking the same answer.

ie. Check using sed ':a;/0$/{N;s/\n//;ba}' verbatim

sed ':a;/0$/{N;s/\n//;ba}'
 does
 no one
 ie. 0
 people,
 try
 nothing,

 ie. 0
 things,
 any more,
 ie. 0
 tests?

          (^D aka eot 004 ctrl-D ␄  ... bash generate via: echo ^V^D)

which will not give (do the test ;):

 does no one ie. 0
 people, try nothing, ie. 0
 things, any more, ie. 0
 tests?          (^D aka eot 004 ctrl-D ␄  ... bash generate via: echo ^V^D)

To get this use:

sed 'H;${z;x;s/\n//g;p;};/0$/!d;z;x;s/\n//g;'

or:

sed ':a;/0$/!{N;s/\n//;ba}'

not:

sed ':a;/0$/{N;s/\n//;ba}'

Notes:

sed 'H;${x;s/\n//g;p;};/0$/!d;z;x;s/\n//g;'

does not use branching and
is identical to:

sed '${H;z;x;s/\n//g;p;};/0$/!{H;d;};/0$/{H;z;x;s/\n//g;}'

H commences all sequences
d short circuits further script command execution on the current line and starts the next cycle so address selectors following /0$/! can only be /0$/!! so the address selector of
/0$/{H;z;x;s/\n//g;} is redundant and not needed.
if a line does not end with 0 save it in hold space
/0$/!{H;d;}
if a line does end with 0 save it too and then print flush (double entendre ie. purged and lines aligned)
/0$/{H;z;x;s/\n//g;}
NB ${H;z;x;s/\n//g;p;} uses /0$/ ... commands with an extra p to coerce the final print and with a now unnecessary z (to empty and reset pattern space like s/.*//)

score 3 · Answer 4 · answered Oct 21 '11 at 17:44

A typically cryptic Perl one-liner:

perl -pe 'BEGIN{$/="0\n"}s/\n//g;$_.=$/'

This uses the sequence "0\n" as the record separator (by your question, I'm assuming that every line should end with a zero). Any record then should not have internal newlines, so those are removed, then print the line, appending the 0 and newline that were removed.

Another take to your question would be to ensure each line has 17 pipe-separated fields. This does not assume that the 17th field value must be zero.

awk -F \| '
    NF == 17 {print; next}
    prev {print prev $0; prev = ""}
    {prev = $0}
'

score 2 · Answer 5 · answered Oct 21 '11 at 16:08

2

if ends with 0 store, remove newline..

sed '/0$/!N;s/\n//'

answered Oct 21 '11 at 16:08

Karoly Horvath

94,607
11
117
176

I don't think this quite works. If I understand the question, a compete lack of zeros should give one big line. But when I try `$ ls | sed '/0$/!N;s/\n//' I find that it only joins lines in pairs. – Eric Wilson Oct 21 '11 at 17:19
Eric, you are correct. if there is NO 0 then I need to remove the newline – veilig Oct 21 '11 at 17:36
if that's the case check what @ninjalj submitted. – Karoly Horvath Oct 21 '11 at 18:51

sed join lines together

5 Answers5

Linked

Related