-2

I'd like to insert multiple lines above a target line in a file by using sed.

The file.txt below contains one line "target line". My initial version is using the single quote:

sed '/target line/ i\
     inserted line1;\
     inserted line2;\
     inserted line3;' file.txt

The result is:

    inserted line1;
    inserted line2;
    inserted line3;
target line

This version works as expected that the newline at the end of each line is escaped by \ to a literal newline instead of a command terminator. Refer to here.

Then I'd like to use shell variable in the replacement string, so I tried to use double quotes to enable the variable expansion:

sed "/target line/ i\
     inserted line1;\
     inserted line2;\
     inserted line3;" file.txt

But this time the newline and the first four spaces disappeared:

inserted line1;    inserted line2;    inserted line3;
target line

How do I correctly insert a newline in double quotes here?

taoistmiao
  • 29
  • 5
  • 5
    Please, do not show text with images. They are not searchable, not copy-paste-able and much heavier than needed. Moreover they affect accessibility negatively. Please copy-paste the text in your question and [format it properly](https://stackoverflow.com/help/formatting), instead. – Renaud Pacalet Jun 29 '22 at 05:33
  • IMHO it is best to encapsulate sed commands in single quotes and when needed, punch-a-hole through to the shell beneath by closing the existing command with a single quote, doing some shell e.g showing a shell variable `"${someVariable}"` and then opening another. Using double quotes to encapsulate sed commands leads to weird side effects which can be difficult to diagnose or even realise have happened. BTW to include a single quote, punch-a-hole and in the hole enter `\'`. – potong Jun 29 '22 at 11:12
  • @RenaudPacalet Hi, thank you for the reminder and reference. I'll edit the question to remove the images. – taoistmiao Jun 29 '22 at 14:52

3 Answers3

5

With single-quotes:

The backslash followed by the newline are transmitted as-is to sed. Then sed actually uses the backslash to escape the raw newline character into the string rather than terminating the command. See:

$ printf %s 'hello\
world' | hexdump -C

Which clearly shows the backslash 5c followed by the newline 0a contained in the string.

00000000  68 65 6c 6c 6f 5c 0a 77  6f 72 6c 64              |hello\.world|
0000000c

With double-quotes:

The backslash has special meaning in double-quotes. It causes the following newline character to be interpreted as a string continuation character. The consequence is that neither the backslash or the newline are contained in the string and so not seen by sed.

$ printf %s "hello\
world" | hexdump -C

The string is continued without backslash and without newline:

00000000  68 65 6c 6c 6f 77 6f 72  6c 64                    |helloworld|
0000000a

EDIT:

  • Precised that sed actually uses the backslash to escape the following newline character as @dan pointed out.
Léa Gris
  • 17,497
  • 4
  • 32
  • 41
  • Hi, thank you for the insights. Based on the explanation, can I say the third example in my question works because the newline `\n` I typed "explicitly" in the string is not captured by the sed as a command terminator, but a character in the string as-is, and the `\\` after it causes the "implicit" newline become a line continuation instead of a terminator as well? – taoistmiao Jun 29 '22 at 15:56
  • 1
    @taoistmiao in the 3ʳᵈ example, you don't need the backslash at the begging of the 2ⁿᵈ line. Otherwise you got it quite wrong. In double-quote context `\n` is invalid and should be written `\\n`; then it would end-up transmitted as `\n` literally to sed, which by chance you use GNU sed which know to decode `\n` into an actual newline. Because of double-quote, the last backslash at the end of the line is decoded by the shell as a string continuation and so is never seen by sed. – Léa Gris Jun 29 '22 at 16:06
2

In double quotes, backslash escapes these characters (only):

$`"\

and newline character. So eg echo "\$" prints $.

To preserve the backslash in double quotes, escape it with another backslash:

 sed "/target line/ i\\
     inserted line1;\\
     inserted line2;\\
     inserted line3;" file.txt
dan
  • 4,846
  • 6
  • 15
  • Poster wanted to preserve newlines, not those backslashes. The right answer to this is just remove the backslashes at all, sed just ignores them unless you need to pass a backslash to sed to escape Regex grammar elements. – Léa Gris Jun 29 '22 at 10:49
  • @LéaGris That's not correct. The (single) backslashes are required to escape the raw new lines in the text to be inserted. OP (presumably) wishes to insert 3 lines before the matching line. – dan Jun 29 '22 at 11:17
  • 2
    @LéaGris From POSIX: "The argument _text_ shall consist of one or more lines. Each embedded in the text shall be preceded by a . Other characters in text shall be removed, and the following character shall be treated literally." _text_ refers to the argument to the `i` command. Implentations like GNU sed also expand `\t` and so on. But raw newlines here must be escaped, or they are treated as command terminators – dan Jun 29 '22 at 11:27
  • Hi, thank you for the answer. This is the rational solution to my original problem :) – taoistmiao Jun 29 '22 at 16:10
0

If it is not imperative to use sed, there are much easier ways of doing this.

#!/bin/sh

var1=foo
var2=bar
var3=baz

cat > stack <<EOF
${var1}
${var2}
${var3}
EOF

You can likewise create a here document for use with ed, to insert lines before existing ones.

cat > ed1 <<EOF
1i
line1
line2
line3
.
wq
EOF

ed -s existing_file < ed1

That will insert those three lines before line 1 in your existing file, so that what was the first line will become line 4.

petrus4
  • 616
  • 4
  • 7