unix command line cut and remove new line, unwrap next line

Question

I have a text file that has line wraps that I need to get rid of. It has multiple blocks with one blank line in between each block. It is formatted like this:

    A7233334  PALLETS     2                       11.000     EACH
 2331      Storm Supply                             247.5000        2,722.50       2,722.50
 4144      Gentro Sales                             225.0000        2,475.00       2,475.00
 5155      Wacca Outfit                             225.0000        2,475.00       2,475.00
 6661      Acme Warehouse &
           Fuller Supply                            225.0000        2,475.00       2,475.00
 1661      McHugh & Donaldson                       225.0000        2,475.00       2,475.00
 2344      K-10 &
           Dunn Co                                  225.0000        2,475.00       2,475.00
 6662      Ronaldson and Son Co.                    225.0000        2,475.00       2,475.00


    A7233335  CRATES     2                        1.000     EACH
 2331      Storm Supply                             147.5000        1,722.50       1,722.50
 4144      Gentro Sales                             125.0000        1,475.00       1,475.00
 5155      Wacca Outfit                             125.0000        1,475.00       1,475.00
 6661      Acme Warehouse &
           Fuller Supply                            125.0000        1,475.00       1,475.00
 1661      McHugh & Donaldson                       125.0000        1,475.00       1,475.00
 2344      K-10 &
           Dunn Co                                  125.0000        1,475.00       1,475.00
 6662      Ronaldson and Son Co.                    125.0000        1,475.00       1,475.00

I need to unwrap the lines with the line breaks so it outputs to a new file like this:

    A7233334  PALLETS     2                       11.000     EACH
 2331      Storm Supply                             247.5000        2,722.50       2,722.50
 4144      Gentro Sales                             225.0000        2,475.00       2,475.00
 5155      Wacca Outfit                             225.0000        2,475.00       2,475.00
 6661      Acme Warehouse &          Fuller Supply  225.0000        2,475.00       2,475.00
 1661      McHugh & Donaldson                       225.0000        2,475.00       2,475.00
 2344      K-10 &  Dunn Co                          225.0000        2,475.00       2,475.00
 6662      Ronaldson and Son Co.                    225.0000        2,475.00       2,475.00


    A7233335  CRATES     2                        1.000     EACH
 2331      Storm Supply                             147.5000        1,722.50       1,722.50
 4144      Gentro Sales                             125.0000        1,475.00       1,475.00
 5155      Wacca Outfit                             125.0000        1,475.00       1,475.00
 6661      Acme Warehouse &  Fuller Supply          125.0000        1,475.00       1,475.00
 1661      McHugh & Donaldson                       125.0000        1,475.00       1,475.00
 2344      K-10 &  Dunn Co                          125.0000        1,475.00       1,475.00
 6662      Ronaldson and Son Co.                    125.0000        1,475.00       1,475.00

Any ideas?

score 1 · Answer 1 · edited May 23 '17 at 12:18

This awk one liner can do it:

$ awk '{if ($0 ~ /&$/) {getline a; print $0,a} else {print }}' file

it gets the lines ending with & and joins them with the next one. Otherwise just prints the line.

Test

$ awk '{if ($0 ~ /&$/) {getline a; print $0,a} else {print }}' file
    A7233334  PALLETS     2                       11.000     EACH
 2331      Storm Supply                             247.5000        2,722.50       2,722.50
 4144      Gentro Sales                             225.0000        2,475.00       2,475.00
 5155      Wacca Outfit                             225.0000        2,475.00       2,475.00
 6661      Acme Warehouse &            Fuller Supply                            225.0000        2,475.00       2,475.00
 1661      McHugh & Donaldson                       225.0000        2,475.00       2,475.00
 2344      K-10 &            Dunn Co                                  225.0000        2,475.00       2,475.00
 6662      Ronaldson and Son Co.                    225.0000        2,475.00

To delete multiple spaces or to make column, you can use | tr -s ' ' and | column.

Update

To delete the leading space we can do this:

awk '{if ($0 ~ /&$/) {getline a; gsub(/^[ \t]+|[ \t]+$/,"",a); print $0,a} else {print }}' file

Based on https://stackoverflow.com/a/8766188/1983854.

Test:

$ awk '{if ($0 ~ /&$/) {getline a; gsub(/^[ \t]+|[ \t]+$/,"",a); print $0,a} else {print }}' file
    A7233334  PALLETS     2                       11.000     EACH
 2331      Storm Supply                             247.5000        2,722.50       2,722.50
 4144      Gentro Sales                             225.0000        2,475.00       2,475.00
 5155      Wacca Outfit                             225.0000        2,475.00       2,475.00
 6661      Acme Warehouse & Fuller Supply                            225.0000        2,475.00       2,475.00
 1661      McHugh & Donaldson                       225.0000        2,475.00       2,475.00
 2344      K-10 & Dunn Co                                  225.0000        2,475.00       2,475.00
 6662      Ronaldson and Son Co.                    225.0000        2,475.00       2,475.00


    A7233335  CRATES     2                        1.000     EACH
 2331      Storm Supply                             147.5000        1,722.50       1,722.50
 4144      Gentro Sales                             125.0000        1,475.00       1,475.00
 5155      Wacca Outfit                             125.0000        1,475.00       1,475.00
 6661      Acme Warehouse & Fuller Supply                            125.0000        1,475.00       1,475.00
 1661      McHugh & Donaldson                       125.0000        1,475.00       1,475.00
 2344      K-10 & Dunn Co                                  125.0000        1,475.00       1,475.00
 6662      Ronaldson and Son Co.                    125.0000        1,475.00       1,475.00

Unfortunately I have lines in between items as the file will have multiple blocks like that with spaces in between that have to stay. Is there a way to look and skip around those full blank lines and still output. Your output works just kills the full blank lines in between each block. — stickfigurefred, Jun 04 '13 at 16:03
Please update your question with some examples so I can guess an update on the answer. — fedorqui, Jun 04 '13 at 16:05
Updated the question with example. Hope this explains it a little better. There should be one blank line in between each block for an item like Pallets and Crates. — stickfigurefred, Jun 04 '13 at 16:49
OK I see. Check my updated answer, this is the further I could arrive. — fedorqui, Jun 04 '13 at 18:05
That works perfect except for not leaving a blank line in between each block. Is there something that can check for a full blank line and keep it? — stickfigurefred, Jun 04 '13 at 18:42
Mmmm it does get a full blank line between each big block. Maybe I am not understanding it but in my output it does. — fedorqui, Jun 04 '13 at 19:03
Hmm there should be a full spaced line above A7233335 Crates 2. It seems to be clearing out all blank lines on my side. I'm testing under Fedora and RedHat to see if there is a diff and both give me same results. — stickfigurefred, Jun 04 '13 at 20:03
My bad fedorqui, I was reading in | column which removed the spacings. Your update was working correctly. Thank you so much! — stickfigurefred, Jun 04 '13 at 20:10
Haha no problem, now I feel better because I was still wondering what could be wrong :D Cheers! — fedorqui, Jun 04 '13 at 20:35

Nitin4873 · Accepted Answer · 2013-06-05T03:28:02.583

Finally ....... got it right ......... :D

code :

i=1;
sed -i '/^$/d' zlist3 ;  ## this is the reason it took me quite some time :( , the damm thing messes up the code if there are blank lines in between in input file ....

cat zlist3 |  nl | grep "\&$" | tr -s ' ' | cut -c1-4 >> ztest5.csv ;
limit=`cat zlist3 | wc -l`;

while [ $i -le $limit ]
do
grep $i ztest5.csv ;
if [ $? -ne 0  ]
then
  sed -n "${i}p"  zlist3 | tr -s ' ' >> zlist4 ;
else
  tmp1=`sed -n "${i}p"  zlist3`;
  i=$(($i+1));
  tmp2=`sed -n "${i}p"  zlist3`;
  echo "$tmp1 $tmp2" | tr -s ' ' >> zlist4 ;
fi

 i=$(($i+1));
done

as you can see the code is based on the sequence number ...........

output :

Kaizen ~

$ cat zlist4
 A7233334 PALLETS 2 11.000 EACH 
 2331 Storm Supply 247.5000 2,722.50 2,722.50
 4144 Gentro Sales 225.0000 2,475.00 2,475.00
 5155 Wacca Outfit 225.0000 2,475.00 2,475.00
 6661 Acme Warehouse & Fuller Supply 225.0000 2,475.00 2,475.00
 1661 McHugh & Donaldson 225.0000 2,475.00 2,475.00
 2344 K-10 & Dunn Co 225.0000 2,475.00 2,475.00
 6662 Ronaldson and Son Co. 225.0000 2,475.00 2,475.00
 A7233335 CRATES 2 1.000 EACH
 2331 Storm Supply 147.5000 1,722.50 1,722.50
 4144 Gentro Sales 125.0000 1,475.00 1,475.00
 5155 Wacca Outfit 125.0000 1,475.00 1,475.00
 6661 Acme Warehouse & Fuller Supply 125.0000 1,475.00 1,475.00
 1661 McHugh & Donaldson 125.0000 1,475.00 1,475.00
 2344 K-10 & Dunn Co 125.0000 1,475.00 1,475.00
 6662 Ronaldson and Son Co. 125.0000 1,475.00 1,475.00

....... i dont care about the question anymore , completely enjoyed the coding ..... :D

oh by the way i hope this helps !!

Thanks for your input Kaizen! A little more then I needed but appreciate your thought on it. — stickfigurefred, Jun 04 '13 at 20:20
no problem , i completely enjoyed the coding part .....by the way my name's NSD , Kaizen is my Laptop's name. I will convey your message to my Kaizen :P ? — Nitin4873, Jun 05 '13 at 03:24

unix command line cut and remove new line, unwrap next line

2 Answers2

Test

Update