Convert merge line batch script to shell

Question

I recently asked a question on the same forum and found a solution for it. Unfortunately, it has to be converted to UNIX. The issue was to merge lines from a csv file. Each line should end with a semi-colon (;) and if it doesn't combine the next line into it until find a semicolon again.

the solution that worked for me was

@echo off
setlocal disableDelayedExpansion
set "ln="
for /f delims^=^ eol^= %%i in (myfile.txt) do (
    set "var=%%i"
    setlocal enableDelayedExpansion
    if "!var:~-1!"==";" (echo !var!>>temp.csv) else (<nul set /p ="!var!">>temp.csv)
    endlocal
)

Would it be possible to convert to UNIX script?

Link to the original post is: Merge line with the next line if last character is a semicolon using batch file

There's no such thing as "Unix scripts." There are scripting languages, shells, pipeline utilities, and a multitude of programming languages. Do some research and write some code, then you'll be in a better position to ask good Stack Overflow questions. — Todd A. Jacobs, Jan 17 '13 at 17:01

score 0 · Accepted Answer · answered Jan 17 '13 at 17:02

next time if you want to do text processing, don't go to windows. :D

given that you have:

A;1;abc;<xml/>;
;2;def;<xml
>hello world</xml>;
;3;ghi;<xml/>;

your rule is:

I need to combine lines such that if the line doesn't end end with a semicolon (;), combine the next line into the current line.

awk '!/;$/{l=l""$0;next;}{print l""$0;l=""}' file

test

kent$  echo "A;1;abc;<xml/>;
;2;def;<xml
>hello world</xml>;
;3;ghi;<xml/>;"|awk '!/;$/{l=l""$0;next;}{print l""$0;l=""}'
A;1;abc;<xml/>;
;2;def;<xml>hello world</xml>;
;3;ghi;<xml/>;

and it works in this case too:

kent$  cat ttt
A;1;abc;<xml/>;
;2;def;<xml
>h
e
l
l
o w
o
rld<
/xm
l>;
;3;ghi;<xml/>;

kent$  awk '!/;$/{l=l""$0;next;}{print l""$0;l=""}' ttt
A;1;abc;<xml/>;
;2;def;<xml>hello world</xml>;
;3;ghi;<xml/>;

Thanks @Kent. I am getting an error about the size of the data as each row of my CSV file is quite large. I found that I might need to use perl. is it be possible to use the same regex with perl? — Junaid, Jan 18 '13 at 12:26

score 0 · Answer 2 · answered Jan 17 '13 at 17:06

0

sed would be my option.

sed ':a;/[^;]$/{N;s/\n//;ta}' x.txt

sed reads in each line and for each one sets a tag (:a) then checks to see if the line is missing the ; (/[^;]$/) if it doesn't have the ; N reads in the next line and appends it to the current line with a new line seperating them. s/\n// removes the newline ta checks to see if the substitution was successfull and if so jumps to the :a tag to start the check again

answered Jan 17 '13 at 17:06

peteches

3,447
1
13
15

Thanks @peteches. I am getting an error about the size of the data as each row of my CSV file is quite large. I found that I might need to use perl. is it be possible to use the same regex with perl? – Junaid Jan 18 '13 at 12:35

Convert merge line batch script to shell

2 Answers2