How to replace only first matching block using sed command

Question

I have 2 files and would like to replace the 1st matching block of text from file1 (matching criteria start indicator as <<EOF and End of block criteria is EOF) with sed from another file2 data.

Example:

file1

resource "policy" "temp1" { 
 policy = << EOF
 {
  policy1-data
 }
 EOF
}

resource "policy" "temp2" { 
 policy = << EOF
 {
  policy2-data
 }
 EOF
}

file2

{
  modified-policy-data
}

output file

resource "policy" "temp1" { 
 policy = << EOF
 {
  modified-policy-data
 }
 EOF
}

resource "policy" "temp2" { 
 policy = << EOF
 {
  policy2-data
 }
 EOF
}

I tried below sed command but that replaces all occurrences of matching criteria in file1 from file2 data where I would like to replace only first matching block occurrence and rest keep same as is from file1

lead='<<EOF'
tail='EOF'
sed -nie '/'"$lead"'/ { p r file2 :a n /'"$tail"'/ { p b } ba }p' file1

What is the **actual** command you tried? The one you show is not valid sed syntax. — jhnc, Apr 13 '23 at 01:59
nit-pick - `head` rather than `lead` is usually used as the opposite of `tail`, using `lead` instead of `head` might cause some confusion at some point, especially given how similar they are. — Ed Morton, Apr 13 '23 at 13:18

Jetchisel · Answer 1 · 2023-04-13T03:58:24.090

1

Not sed but if ed is available/acceptable with your given input/data, something like:

The script.ed name it to your own hearts content.

0r file2
kx
2m/^.*<< EOF/;/^.*EOF/;?[ ]*{?
+1d
1,'xd
,p
Q

Now run

ed -s file1 < script.ed

Output

resource "policy" "temp1" { 
 policy = << EOF
 {
  modified-policy-data
 }
 EOF
}

resource "policy" "temp2" { 
 policy = << EOF
 {
  policy2-data
 }
 EOF
}

In one-line

printf '%s\n' '0r file2' 'kx' '2m/^.*<< EOF/;/^.*EOF/;?[ ]*{?' '+1d' "1,'xd" ,p Q | ed -s file

with variables in a shell script, something like:

file1

resource "policy" "temp1" { 
 policy = << EOF
 {
  policy1-data
 }
 EOF
}

resource "policy" "temp2" { 
 policy = << EOF
 {
  policy2-data
 }
 EOF
}

file2

{
  modified-policy-data
}

foo
bar
baz

The script.

#!/bin/sh

lead='<< EOF'
tail='EOF'

ed -s file1 <<-EOD
  0r file2
  kx
  2m/^.*$lead/;/^.*$tail/;?[ ]*{?
  +1d
  1,'xd
  ,p
  Q
EOD

OTOH, your data/files/input looks like tcl

Change Q to w if in_place editing is needed.
Remove the ,p to silence the output.
See GNU ed
See POSIX ed

edited Apr 13 '23 at 03:58

answered Apr 13 '23 at 02:36

Jetchisel

7,493
2
19
18

this seems to break if file2 does not contain 3 lines – jhnc Apr 13 '23 at 02:48
1

`printf '/<< EOF/r file2\n,.p\n/EOF/,$p\nQ\n' | ed -s file1` ? – jhnc Apr 13 '23 at 03:58
`printf '%s\n' '/<< EOF/r file2' '.+;/EOF/-d' ,p Q | ed -s file1` – Jetchisel Apr 13 '23 at 04:41
that does allow replacing Q with w but anything that uses `+` can break: eg. if file1 initially has no lines between lead and tail – jhnc Apr 13 '23 at 04:53
Good catch! got me thinking... – Jetchisel Apr 13 '23 at 08:28
1

`printf '%s\n' '/<< EOF/r file2' '.t.' '.;/EOF/-1d' ,p Q | ed -s file1` ? – Jetchisel Apr 13 '23 at 08:51
1

looks good to me! – jhnc Apr 13 '23 at 19:32

jhnc · Answer 2 · 2023-04-14T14:56:18.317

1

Once you match lead you must never start a new cycle (ie. b)

sed -nie '
  p
  /'"$lead"'/ {
    r file2

    :a
    n
    /'"$tail"'/ ! ba

    :b
    p
    n
    bb
  }
' file1

edited Apr 14 '23 at 14:56

answered Apr 13 '23 at 02:49

jhnc

11,310
1
9
26

William Pursell · Answer 3 · 2023-04-13T12:53:08.680

1

It seems like awk would be easier than sed. Something like:

awk '! seen && /<< EOF$/ { seen=1; print; 
    while( (getline < "file2") > 0 ) { print } 
    while( (getline > 0) && ! /EOF/ ); } 1' file1

edited Apr 13 '23 at 12:53

answered Apr 13 '23 at 03:18

William Pursell

204,365
48
270
300

Ed Morton · Answer 4 · 2023-04-13T13:17:09.537

Using any awk and aligning the new text with the existing EOF:

$ cat tst.awk
NR == FNR {
    new = (NR>1 ? new ORS : "") $0
    next
}
cnt == 1 {
    if ( index($0,tail) ) {
        indent = $0
        sub(/[^[:space:]].*/,"",indent)
        gsub(ORS,ORS indent,new)
        print indent new ORS $0
        cnt++
    }
    next
}
index($0,lead) {
    cnt++
}
{ print }

$ awk -v lead='<< EOF' -v tail='EOF' -f tst.awk file2 file1
resource "policy" "temp1" {
 policy = << EOF
 {
   modified-policy-data
 }
 EOF
}

resource "policy" "temp2" {
 policy = << EOF
 {
  policy2-data
 }
 EOF
}

potong · Answer 5 · 2023-04-14T09:13:46.883

1

This might work for you (GNU csplit):

csplit -s file1 '/EOF/+1' '%EOF%' && cat xx00 file2 xx01 > file3 && rm xx??

Use csplit to split file1 into two and remove the text to replaced by file2.

Concatenate the files above with the replacement file sandwiched between them.

Alternative using GNU sed:

sed -ne '/EOF/!{p;b};p;:a;n;//!ba;e cat file2' -e ':b;p;n;bb' file1

Clean up unwanted files.

edited Apr 14 '23 at 09:13

answered Apr 14 '23 at 00:53

potong

55,640
6
51
83

score 0 · Answer 6 · answered Apr 14 '23 at 10:11

If you want to specify which section to replace by resource name, you could do it like this with GNU sed and a bash function:

repl file1 file2 temp1

Where repl is defined as:

repl() {
  sed -E '
    # Always keep 2 lines in pattern space
    N
    /resource.*"'$3'".*\n.*<< *EOF/ {
      p
      :a
      /^ *EOF$/! {
        N
        s/.*\n//
        ba
      }
      r '$2'
      N
    }
    P
    D' "$1"
}

Testing:

repl file1 file2 temp1

Output:

resource "policy" "temp1" {
 policy = << EOF
{
  modified-policy-data
}
 EOF
}

resource "policy" "temp2" {
 policy = << EOF
 {
  policy2-data
 }
 EOF
}

Or replace temp2:

repl file1 file2 temp2

Output:

resource "policy" "temp1" {
 policy = << EOF
 {
  policy1-data
 }
 EOF
}

resource "policy" "temp2" {
 policy = << EOF
{
  modified-policy-data
}
 EOF
}

How to replace only first matching block using sed command

6 Answers6