Using cat command start each file with new line

Question

I am trying to merge different files using cat command into one file.

cat F_Worker_TEMP_VO.dat F_PERSON_NAME_VO.dat F_PERSON_CITIZENSHIP_VO.dat F_PERSON_LEGISLATIONDATA_VO.dat F_PERSON_RELIGION_VO.dat F_PERSON_PASSPORT_VO.dat F_PERSON_DRIVERSLIC_VO.dat F_PERSON_NATIONALIDE_VO.dat F_PERSON_NATIONALIDE_ADH_TEMP_VO.dat F_PERSON_NATIONALIDE_SSN_TEMP_VO.dat > Worker.dat

where Worker.dat is my output file.

Using the above code the second file gets started at the end of first file data. I want each file data to be started with a new line

I have tried inserting /n, echo in the above code. it's not giving me the output

cat F_Worker_TEMP_VO.dat F_PERSON_NAME_VO.dat F_PERSON_CITIZENSHIP_VO.dat F_PERSON_LEGISLATIONDATA_VO.dat F_PERSON_RELIGION_VO.dat F_PERSON_PASSPORT_VO.dat F_PERSON_DRIVERSLIC_VO.dat F_PERSON_NATIONALIDE_VO.dat F_PERSON_NATIONALIDE_ADH_TEMP_VO.dat F_PERSON_NATIONALIDE_SSN_TEMP_VO.dat > Worker.dat

Sample Expected format Each metadata is the result of different files I am merging. Each metadata should start with a new line

METADATA    Worker  EffectiveStartDate  EffectiveEndDate    StartDate   CategoryCode    ActionCode  BloodType   CorrespondenceLanguage  DateOfBirth TownOfBirth SourceSystemOwner
METADATA    PersonName  PersonNameId    EffectiveStartDate  EffectiveEndDate    PersonId(SourceSystemId)    SourceSystemOwner   SourceSystemId  Title   KnownAs LegislationCode NameType
METADATA    PersonCitizenship   CitizenshipId   PersonId(SourceSystemId)    DateFrom    DateTo  LegislationCode CitizenshipStatus   SourceSystemOwner   SourceSystemId  



Actual
METADATA    Worker  EffectiveStartDate  EffectiveEndDate    StartDate   CategoryCode    ActionCode  BloodType   CorrespondenceLanguage  DateOfBirth TownOfBirth SourceSystemOwner   SourceSystemIdMETADATA  PersonName  PersonNameId    EffectiveStartDate  EffectiveEndDate    PersonId(SourceSystemId)    SourceSystemOwner   SourceSystemId  Title   KnownAs LegislationCode NameTypeMETADATA    PersonCitizenship   CitizenshipId   PersonId(SourceSystemId)    DateFrom    DateTo  LegislationCode CitizenshipStatus   SourceSystemOwner

Currently, there is a gap between each result file. It getting started with the end of previous file result

1. Iterate over files. 2. For each file a) cat it b) print a newline. Alternatively more advanced - input list of files on each line to `xargs` that applies it to `sed` that adds a newline on the last line of the file. — KamilCuk, Oct 03 '19 at 09:56
Your files don't end with newlines? That's arguably broken right there. — Shawn, Oct 03 '19 at 10:04
No, the results are coming are continued with the previous files data. — neha kirtiwar, Oct 03 '19 at 10:07
Eg, metadata tag for personname file starts right after the data of worker file sourcesystem data — neha kirtiwar, Oct 03 '19 at 10:08
METADATA Worker EffectiveStartDate EffectiveEndDate StartDate CategoryCode ActionCode BloodType CorrespondenceLanguage DateOfBirth TownOfBirth SourceSystemOwner SourceSystemId MERGE Worker 02-10-2019 31-12-4712 02-10-2019 PER_EIT HIRE 29-05-1987 TALEO-RECRUIT PIR_NH_PER_225939METADATA PersonName PersonNameId EffectiveStartDate EffectiveEndDate PersonId(SourceSystemId) SourceSystemOwner SourceSystemId Title KnownAs LegislationCode NameType Result file-metadata tag of the second file starts right after the last data of first file — neha kirtiwar, Oct 03 '19 at 10:12
Sure [Bash Loop Over files](https://www.cyberciti.biz/faq/bash-loop-over-file/) — KamilCuk, Oct 03 '19 at 10:13

score 1 · Accepted Answer · answered Oct 03 '19 at 10:15

Quick and dirty, add newlines before each METADATA:

cat F_Worker_TEMP_VO.dat ....dat | sed 's/METADATA/\nMETADATA/g' > Worker.dat

Dirty, add newlines before each METADATA except first:

cat F_Worker_TEMP_VO.dat ....dat | sed 's/\(.\)METADATA/\1\nMETADATA/g' > Worker.dat

Loop:

for file in F_Worker_TEMP_VO.dat ... F_PERSON_NATIONALIDE_SSN_TEMP_VO.dat; do
   cat "${file}"
   echo
done > Worker.dat

score 1 · Answer 2 · answered Oct 03 '19 at 10:23

1

I believe that this StackOverflow answer covers all that you might need: Concatenating Files And Insert New Line In Between Files

Anyways, here is the command you want:

for f in *.dat; do (cat "${f}"; echo) >> finalfile.dat; done

Or use an array if you want to concatenate only a Subset of files in some order. Sed and Awk uses are also in the linked StackOverflow answer.

answered Oct 03 '19 at 10:23

Animesh Sinha

817
2
11
24

This really looks like a duplicate then. – KamilCuk Oct 03 '19 at 10:51
I am pretty sure it is, but I am new here so I believe that I cannot mark it as one. – Animesh Sinha Oct 03 '19 at 14:34

Using cat command start each file with new line

2 Answers2