SED doesn't process multiple commands in {} after d(elete), a(ppend) or c(hange)

Question

sed should process after a matching-pattern multiple commands which are given in braces like {cmd1;cmd2;cmd3}. But in the given code below, all commands followed after d(elite) are ignored.

script.sed

s/^\(interface GigabitEthernet0\)$/\1\/0/
/interface GigabitEthernet0\/0$/{
n       # process next line = 1st line (after match) to be deleted
d       # Should delete '1st line (after match) to be deleted'
n       # process next line = 2nd line to be altered
s/2nd line to be altered/2ND LINE AFTER ALTERATION/
n
s/3rd line to be altered/3RD LINE AFTER ALTERATION/
}

input.txt

interface GigabitEthernet0
  1st line (after match) to be deleted
  2nd line to be altered
  3rd line to be altered
  4th line stays unchanged

sed -i -f script.sed example.txt

Expected output:

interface GigabitEthernet0/0
  2ND LINE AFTER ALTERATION
  3RD LINE AFTER ALTERATION
  4th line stays unchanged

Effective output:

interface GigabitEthernet0/0
  2nd line to be altered
  3rd line to be altered
  4th line stays unchanged

As seen in the output above, line 1st line (after match) to be deleted has been effectively deleted. But all following lines (2nd, 3rd, 4th) are not substituted.

BTW: Commands like a(ppend) or c(hange) behaves in the same manner; all followed commands are ignored.

pluse-uno for well formullated Q. Keep posting, but learn `awk` ;-) — shellter, May 19 '19 at 22:50

Ed Morton · Accepted Answer · 2019-05-20T04:50:16.947

sed is for doing s/old/new, that is all. Just use awk, e.g. with GNU awk for inplace editing (just like you're doing with GNU sed) and the switch statement:

$ cat tst.awk
/^interface GigabitEthernet0$/ { lineNr=1 }
lineNr%5 {
    switch (lineNr++) {
    case 1: $0 = $0 "/0";               break
    case 2: next;                       break
    case 3: sub(/2nd line/,"WHATEVER"); break
    case 4: $0 = toupper($0);           break
    }
}
{ print }

$ cat file
interface GigabitEthernet0
  1st line (after match) to be deleted
  2nd line to be altered
  3rd line to be altered
  4th line stays unchanged

$ awk -i inplace -f tst.awk file

$ cat file
interface GigabitEthernet0/0
  WHATEVER to be altered
  3RD LINE TO BE ALTERED
  4th line stays unchanged

And just in case you really do just want to print new lines instead of modify the existing ones in the range after you match your regexp (like in @AlexHarvey's sed answer), then that'd just be:

awk '/interface GigabitEthernet0/ {
    print $0 "/0\n  2ND LINE AFTER ALTERATION\n  3RD LINE AFTER ALTERATION"
    c = 4
}
!(c&&c--)' file
interface GigabitEthernet0/0
  2ND LINE AFTER ALTERATION
  3RD LINE AFTER ALTERATION
  4th line stays unchanged

See the list of common awk idioms at https://stackoverflow.com/a/17914105/1745001 for what c&&c-- does.

Note that you don't have to specify the initial regexp multiple times, you don't have to worry about what chars might show up in your replacement text, and you can just give a number for how many lines are impacted you don't need to write the same character that number of times.

oguz ismail · Answer 2 · 2019-05-19T17:37:19.870

As it is stated in the POSIX sed specification, the command d deletes the pattern space and starts the next cycle.

I tweaked your script a bit to get desired output:

/^interface GigabitEthernet0$/{
s//&\/0/
n 
N
s/.*\n//
s/2nd line to be altered/2ND LINE AFTER ALTERATION/
n
s/3rd line to be altered/3RD LINE AFTER ALTERATION/
}

It works, but also shows us that sed is not the right tool for this job. So, as Ed suggested, use awk instead.

Alex Harvey · Answer 3 · 2019-05-20T03:37:25.047

1

As noted by others, the immediate issue is that the d command not only deletes, but also immediately ends the cycle, i.e. stops processing further commands.

But, there is a simpler sed solution available to you here. You seem to have over-complicated this, based on your apparent requirement.¹

# test.sh

l1="interface GigabitEthernet0/0"
l2="  2ND LINE AFTER ALTERATION"
l3="  3RD LINE AFTER ALTERATION"

replacement="$l1\n$l2\n$l3"

sed '/interface GigabitEthernet0/ {N;N;N; s!.*!'"$replacement"'!}' example.txt

Output:

interface GigabitEthernet0/0
  2ND LINE AFTER ALTERATION
  3RD LINE AFTER ALTERATION
  4th line stays unchanged

Further explanation:

on matching /interface GigabitEthernet0/,
slurp the next 3 lines into pattern space (N;N;N),
and then just replace the whole thing with the lines you actually want.
Noting that you need to replace sed's delimiter / with a character you trust not to appear in the input stream. I chose !.

¹ And while I think there is an elegance to Ed's AWK solution, it is trivial to use sed here, where you just need to read forwards in a file by a few lines and do a s/// replacement.

edited May 20 '19 at 03:37

answered May 20 '19 at 02:41

Alex Harvey

14,494
5
61
97

1

Thanks @EdMorton, yes off-by-one error! We slurp 3 lines in to replace a total of 4. Good pickup. – Alex Harvey May 20 '19 at 03:38
Yes, @EdMorton, `N;N;N` aka `for (i=1; i<=3; i++) N` ;-) – Alex Harvey May 20 '19 at 03:42
You know that approach isn't sustainable, though, right? I mean if you had a block of 20 lines are you really going to write 20 `N`s? It's also fragile and will fail given various characters in `$replacement` -- you already had to use `!` as the delimiter instead of `/` because `$replacement` contains a `/` and now you have to be careful that it doesn't get a `!` added to it, nor a `\1` or `&` and maybe other chars. And it wouldn't work at all if you had to modify any of the lines instead of replacing then all en-masse. It's just not a good way to approach such a conceptually simple problem. – Ed Morton May 20 '19 at 03:46
@EdMorton but now you are arguing against your own rule to use sed to do `s/old/new/`. If I have a block of 20 lines I probably won't do this (i.e. obviously, I am using Bash variables there only to improve readability). At the moment, there is more effort involved in reading and writing the AWK solution than this simple sed solution, at least IMO. The cost of rewriting a 1-line script is really negligible. I just don't see a problem here. – Alex Harvey May 20 '19 at 03:54
I used to always say s/old/new on single lines. Now with `-z` it's the same idea but not a single line any more so I'm not really sure exactly how to say it. My point really is though that the only sed commands you should use are `s, g, and p (with -n)`. The problem is the same one I mentioned a few days ago - if you waste your time learning how to cobble together a sed scripts to do one specific variation of an extremely common class of tasks then you miss the opportunity to learn how to write an awk script that could be used for all similar tasks. – Ed Morton May 20 '19 at 04:01
And lets be honest here - the OP doesn't REALLY want to just replace 4 lines after one is found (which would be absolutely trivial, more robust, more efficient and briefer in awk than your sed solution), she wants to modify the 4 lines in different ways, it was just a bad example with the only real clues at what the OP really wants being the subject `SED doesn't process multiple commands in {}` and the text in the example `...to be altered`. – Ed Morton May 20 '19 at 04:02
@EdMorton, I personally do agree that learning all about the ins and outs of branching and flow control in sed is a waste of time. But I don't see why you're so strongly against a basic understanding of sed's grammar and its commands, if the end result is easy to write and understand. – Alex Harvey May 20 '19 at 04:13
It's just a missed learning opportunity. If you have small tasks to do and you learn how to do them using awk then when you have a big task to do you'll already be skilled with the basics and be able to effectively tackle that big task. If instead you spend your time on the small tasks learning how to write a bunch of arcane sed runes then when you do have that bigger task to do you will probably try to tackle it with sed and end up with an incomprehensible, fragile, inefficient, buggy non-portable mess. Or you'll tackle it with awk and have the same outcome as you never learned the basics. – Ed Morton May 20 '19 at 04:22
btw I added an awk equivalent of your sed script to the bottom of my answer just as an FYI. – Ed Morton May 20 '19 at 04:23
@EdMorton it took me 2 days to learn sed. I kid you not. AWK is a much longer journey (when you then consider style, idiom etc). But I do think the fact I'm a vim user makes thinking in a sed-like way more intuitive than it otherwise would be. – Alex Harvey May 20 '19 at 04:27
wrt `AWK is a much longer journey` - exactly. If instead of writing sed scripts you had spent that time writing awk scripts just think how much further along you'd be in that journey. It's like learning to type properly instead of with 2 fingers or learning a new language when you move to a country where many people speak yours as a second language - it takes a bit of effort initially but is well worth it in the end. – Ed Morton May 20 '19 at 04:32
@EdMorton, to be fair, you make a pretty good point, and suggest a good reason to practice always using AWK instead of sed. But even so, it's still only true if learning AWK to such a level of fluency is likely to be important. You could make the same argument for any language. Why not always use Perl? I continue to believe that if your primary use case is quickly getting things done rather than writing maintainable code in a code base, sed is a pretty good quick option. – Alex Harvey May 20 '19 at 05:27
wrt `Why not perl` - 1) because it doesn't come as standard on all UNIX boxes (I spent 30 years working on systems that don't have it and it can't be installed on) and 2) because https://www.zoitz.com/archives/13. wrt using sed - once you learn awk if it's anything more than s/old/new then you can write it as quickly in awk as you can in sed and chances are it'll be clearer, more portable, more robust, more efficient, etc. I think I've preached enough though. All the best! – Ed Morton May 20 '19 at 12:57

SED doesn't process multiple commands in {} after d(elete), a(ppend) or c(hange)

3 Answers3