6

Let's say I have a file called original.txt with this content:

red
blue
water
food
tree
gray
white

Also I have a file called new.txt with this content:

green
black
yellow
purple

Now I want to write a script that replaces the lines between blue and gray in original.txt with the contents of new.txt, so it gives me this result:

red
blue
green
black
yellow
purple
gray
white

I wrote this piece of code for the purpose (the name of the new file is not always the same so it's stored in a variable):

newtext="new.txt"

sed -i "/blue/,/gray/{
    r $newtext
    d
}" original.txt

However, when running it I get this nonsense instead:

red
green
black
yellow
purplegreen
black
yellow
purplegreen
black
yellow
purplegreen
black
yellow
purplegreen
black
yellow
purplewhite

What am I doing wrong?

Stealth
  • 155
  • 2
  • 9
  • You only want to insert the lines from the second file once, not every time you encounter a line in the range. – potong Oct 04 '17 at 22:32

3 Answers3

8

This might work for you (GNU sed):

sed '/blue/,/gray/!b;//!d;/blue/r file2' file1

For the range of lines between blue and gray, delete lines that do not match either the first or the last lines in the range and read the lines to insert before the last line of the range.

EDIT:

The first sed command /blue/,/grey/!bmatches a range of lines i.e. the lines that range between a line containing blue upto and including a line containing gray. The !b part means if the lines are not in that range then break out of the sed commands i.e. do not do any further sed processing for these lines, just print as normal.

The sed commands following will only affect those lines that are in the range between blue and gray.

The second command //!d means: delete those lines that do not match either the start/end of the range i.e. blue or gray. The // uses the regexp from a previous /.../ command. N.B. a delete command terminates any further sed processing for that line.

The sed commands following will only affect lines that containing either blue or gray.

The third command matches a line containing blue and reads in lines from file2.

N.B. the lines containing blue and grey are processed by sed naturally and printed before the next line is read into the pattern space as are the lines not between blue and gray.

An alternative:

sed '/blue/,/gray/!b;//!d;/gray/e cat file2' file1

And another:

sed -ne '/blue/{p;r file2' -e ':a;n;/gray/!ba};p' file1
potong
  • 55,640
  • 6
  • 51
  • 83
  • Thank you! It worked pretty good, I just needed to add `//a` in order to append a new line because `purple` and `gray` were getting merged. Would you mind explaining what does each command do though? Also, is there a way to avoid writing `blue` twice in the instructions? – Stealth Oct 04 '17 at 23:38
  • If you can add how these `sed` commands work, that would be very helpful indeed. – codeforester Oct 05 '17 at 04:29
  • Also want to know what is `//` for command `//!d`, can someone please shed light on it? – CWLiu Oct 05 '17 at 05:30
  • can also use `sed -e '/blue/r file2' -e '/blue/,/gray/{//!d} file1'`... however, both solutions do not strictly check if both `blue` and `gray` lines exist... if there is no `gray`, it will delete all lines after `blue` and so on – Sundeep Oct 05 '17 at 06:14
  • 1
    @CWLiu see https://stackoverflow.com/a/38978201/4082052 and comment section – Sundeep Oct 05 '17 at 06:21
5

sed is for s/old/new/, that is all. For anything else you should be using awk:

$ awk 'NR==FNR{new = new $0 ORS; next} /gray/{f=0} !f{print} /blue/{printf "%s",new; f=1}' new.txt original.txt
red
blue
green
black
yellow
purple
gray
white

The above will work in any awk in any shell on any UNIX box and does not require any of the start/end regexps to be repeated. Very importantly it can also trivially be built upon to do anything else you want now or in future! For example to be able to specify the beginning and ending regexps as arguments would be:

$ awk -v beg='blue' -v end='gray' 'NR==FNR{new = new $0 ORS; next} $0~end{f=0} !f{print} $0~beg{printf "%s",new; f=1}' new.txt original.txt

ad then you can change them:

$ awk -v beg='water' -v end='tree' 'NR==FNR{new = new $0 ORS; next} $0~end{f=0} !f{print} $0~beg{printf "%s",new; f=1}' new.txt original.txt
red
blue
water
green
black
yellow
purple
tree
gray
white

Anything you might want to do is just a simple rearrangement using the same constructs, e.g. to not print the beginning line:

$ awk -v beg='blue' -v end='gray' 'NR==FNR{new = new $0 ORS; next} $0~end{f=0} $0~beg{printf "%s",new; f=1} !f{print}' new.txt original.txt
red
green
black
yellow
purple
gray
white

and similar tweaks to not print the ending line or not print both or do anything else....

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • doesn't actually check if both start and end regexp are matched before replacing... but wasn't mentioned as requirement by OP, so I guess it is not a problem... – Sundeep Oct 05 '17 at 06:20
  • Thank you for the answer. I used sed for reasons of comfort since it's what I always use and has a much shorter and cleaner syntax; also I needed it for a private project so I don't really require portability; and I was sure that this task was possible with sed so I mainly wanted to know where was the error in my code. But I give you the reason, I should start using awk for these cases as it's always better to rely on standard and portable solutions, and I thank you again for explaining me how. :) – Stealth Oct 05 '17 at 22:53
  • 1
    `cleaner syntax`??? You've got to be kidding. I've been using sed for 40 years and usually can't tell if a given script is going to read in a file or summon the god cthulu. All to our own I suppose.. And yeah, you should really learn awk - you'll be amazed how much simpler and more consistent everything becomes. – Ed Morton Oct 05 '17 at 23:14
1

I see you asked for sed help, but ex handles this use-case in a bit more straightforward way. This may work for you (in a shell script):

ex original.txt << EOF_EX
/blue/+1,/gray/-1 d
-1 r new.txt
w edited.txt
q!
EOF_EX

The above opens original.txt for editing, deletes the range of lines one past blue through one before gray, reads the file new.txt, and writes out file edited.txt. The "-1" address beginning the "r" command ensures that the new text is inserted between the two target lines.

Result:

$ cat edited.txt
red
blue
green
black
yellow
purple
gray
white

Alternatively you could replace "w edited.txt" with "wq" and it would save the changes in the original.txt file.

Also, notice the +1 and -1 syntax in the range, which can be very handy. For example if you had wanted to also remove the range-bounding patterns, "blue" and "gray" you could remove the +1 and -1 and it would mean delete the range, including the range-boundary patterns.

Ernest Friedman-Hill
  • 80,601
  • 10
  • 150
  • 186
Leon Shaner
  • 71
  • 8