Merging of two files with substitution

Question

I need to merge two files, adding the contain of the file 2 directly to the end of the file 1. Notable the last line of the first file consist of the phrase "END". I need to remove it and add there the content of the file 2:

file 1:

string 1
string 2
string N
END

file 2:

string 3
string 4
string M

should give me the file 3

string 1
string 2
string N
string 3
string 4
string M

I've tried simply using

cat file1 file2 >> file3

but it did not substitution of the END in the file2. May I use a special option for this?

Does the first file always have `END` as the last line? If so, you can use `cat <(head -n -1 file1) file2` — Sundeep, Jan 12 '22 at 14:51
yes there is always END in the first file, it is like a pattern which should be recognized and substituted by the containts of the file 2. I've jsut tired our sollution but it printed to me only the content of the file 2, so.. — Hot JAMS, Jan 12 '22 at 14:54

anubhava · Accepted Answer · 2022-01-12T15:10:41.320

3

Using sed:

sed -e '${r file2' -e ';$d;}' file1

string 1
string 2
string N
string 3
string 4
string M

If you want to use shell variable instead of file names directly:

sed -i.bak -e "\${r $f2" -e ';$d;}' "$f1"

edited Jan 12 '22 at 15:10

answered Jan 12 '22 at 14:48

anubhava

761,203
64
569
643

thank you! is it possible to do such modification directly in the file 1 without creating the new file using sed -e '${r file2' -e ';$d;}' file1 >> file3 – Hot JAMS Jan 12 '22 at 14:52
@HotJAMS Depends on your `sed` version, but usually you can do `sed -i /^END$/d file1` and it will remove lines containing `END`; IIUC, that's what you're looking for. – daniu Jan 12 '22 at 14:55
1

@HotJAMS: You can use: `sed -i.bak -e '${r file2' -e ';$d;}' file1` to save changes in `file1` – anubhava Jan 12 '22 at 14:55
if I use this sed inside of my bash script, may I use a bash variable within the sed to indicate the path to the file 2 and file1 ? sed -i.bak -e '${r "${path_defined_in_my_bash_script}"/file1.pdb' -e ';$d;}' "${path_defined_in_my_bash_script2}"/file2.pqr – Hot JAMS Jan 12 '22 at 14:59
right, thank you! it just did not work directly in the bash environment compared to the awk sollution mentioned below where file 1 and file2 could be directly defined as the variables .. – Hot JAMS Jan 12 '22 at 15:05

Andre Wildberg · Answer 2 · 2022-01-12T16:13:09.207

1

An approach with awk. It !only! skips ENDs occurring on the last line of file1. It recognizes the last line of file1 by using a helper variable set.

Test with multiple ENDs in a file

% awk 'set==""&&NR!=FNR&&last=="END"{last="";set=1} 
    last!=""{print last} 
    {last=$0} END{print}' file file
string 1
string 2
END
string N
string 1
string 2
END
string N
END

Using file1 and file2

% awk 'set==""&&NR!=FNR&&last=="END"{last="";set=1} 
    last!=""{print last} 
    {last=$0} END{print}' file1 file2
string 1
string 2
string N
string 3
string 4
string M

Data

% cat file
string 1
string 2
END
string N
END

% cat file1
string 1
string 2
string N
END

% cat file2
string 3
string 4
string M

edited Jan 12 '22 at 16:13

answered Jan 12 '22 at 14:58

Andre Wildberg

12,344
3
12
29

1

I think OP wants to remove last line from `file1` not just the line with `END` value – anubhava Jan 12 '22 at 15:11
@anubhava uh, thanks, gonna adjust. – Andre Wildberg Jan 12 '22 at 15:12

potong · Answer 3 · 2022-01-13T01:04:15.173

1

This might work for you (GNU head & cat):

head -n-1 file1 | cat - file2 > file3

Read all but the last line of file1 and concatenate it with file2.

An alternative:

sed -i -e '$r file2' -e '$d' file1

This will append file2 to the end of file1 less the last line of file1 and then replace file1 with the result.

edited Jan 13 '22 at 01:04

answered Jan 12 '22 at 15:39

potong

55,640
6
51
83

score 1 · Answer 4 · answered Jan 13 '22 at 14:54

Adding 1 more variant of tac + awk solution here for FUN.

tac file1 | awk 'FNR==1{system("tac file2");next} 1' | tac

Explanation: Adding detailed explanation for above.

tac file1 |               ##using tac command to read contents from bottom to up and sending its standard output as standard input to awk command here.
awk '                     ##Starting awk program from here.
  FNR==1{                 ##Checking condition if this is first line then do following.
    system("tac file2")   ##Printing file2 bottom to up contents.
    next                  ##next will skip all further contents from here.
  }
  1                       ##Printing rest of lines(passed from tac file1) here.
' | tac                   ##Reversing order again to make it into original order.

markp-fuso · Answer 5 · 2022-01-12T15:27:00.840

Assumptions:

file1 just has the one entry labeled END
the END entry is at the end of file1

Small modification to OP's current cat idea using process substitution to strip the END from file1:

$ cat <(grep -v '^END$' file1) file2 > file3
$ cat file3
string 1
string 2
string N
string 3
string 4
string M

A variation on Andre's awk solution whereby we print everything we see except the file1/END entry:

$ awk 'FNR==NR && /^END$/ {next} 1' file1 file2 > file3
$ cat file3
string 1
string 2
string N
string 3
string 4
string M

Merging of two files with substitution

5 Answers5

Data