1

As the sed implemented in FreeBSD does not support any escape sequence in the replace pattern, I have to use "\'$'\n" to represent a newline and "'$'\n". It does support backslash by "\" though.

Unfortunately, combining the newline with backslash cause error for me. For example, I want to add a line starts with a tab and "line added.\" after every line with "key:\", I wrote:

#!/bin/bash
sed -E 's/^key:\\$/&\'$'\n'$'\tline added.\\/g' file

It thrown me an error like:

sed: 2: "s/^key:\\$/&\
    line add ...": unterminated substitute in regular expression

How do I combine both newline and backslash in the substitute argument of sed?

Thanks a ton!

crackpot
  • 333
  • 1
  • 2
  • 16
  • Can you use a var `my_replacement` and something like `s/^key:\\$/&'"${my_replacement}""/' file`? Or use awk? – Walter A Apr 16 '16 at 19:47
  • 3
    FreeBSD has no `#!/bin/bash` (unless it was manually added), for portability use `#!/bin/sh` or `#!/usr/bin/env bash`. – kdhp Apr 16 '16 at 21:59
  • 1
    [This answer](http://stackoverflow.com/questions/24275070/sed-not-giving-me-correct-substitute-operation-for-newline-with-mac-difference/24276470#24276470) covers a lot of ground, including newlines in BSD sed. – Benjamin W. Apr 17 '16 at 08:01
  • Thanks for reminding me. It is actually a FreeNAS box and there is a soft link on /bin to /usr/local/bin/bash. – crackpot Apr 17 '16 at 14:28

2 Answers2

4

The example is failing because of what appears to be unintended quoting in the last segment.

$'\tline added.\\/g'

The $'...' construct evaluates the entire last section, meaning that it processes \\ leaving the sed command ending in \/g.

To get the intended behavior either terminate the C-style escape, and restart the single quoted string

$'\t''line added.\\/g'

or escape the \\

$'\tline added.\\\\/g'

An alternative would be to enclose the entire command in the $'...' construct.

sed $'s/^key:\\$/&\\\\\\\n\tline added.\\\\/'

A portable alternative is to use literal characters, new-line and tab.
(perhaps with a comment clarifying it as intentional)

sed 's/^key:\\$/&\\\
    line added.\\/'

To avoid duplicates when run multiple times a slightly more complicated script is needed.
(note that n will exit the script if run on the last line)

sed '/^key:\\$/ {
    n
    /^  line added\.\\$/!i\
    line added.\\
}'

$'...' is present in FreeBSD sh, MirBSD ksh, ksh93 (Illumos sh), zsh, and bash

kdhp
  • 2,096
  • 14
  • 15
2

You're trying to use the wrong tool. sed is for simple substitutions on individual lines, that is all. For anything else you should be using awk.

to add a line starts with a tab and "line added.\" after every line with "key:\" would just be:

awk '{print} /key:\\/{print "\tline added.\\"}' file

The above will work in all awks on all OSes.

To NOT do this on a subsequent run that did it previously would be:

awk -v n='\tline added:\\' 'p~/key:\\/ && $0!=n{print n} {print; p=$0}'

e.g.:

$ cat file1
foo
key:\
bar

$ awk -v n='\tline added:\\' 'p~/key:\\/ && $0!=n{print n} {print; p=$0}' file1 > file2

$ cat file2
foo
key:\
        line added:\
bar

$ awk -v n='\tline added:\\' 'p~/key:\\/ && $0!=n{print n} {print; p=$0}' file2
foo
key:\
        line added:\
bar

p for previous, n for new. The above works by just waiting until the line AFTER key:\\ to insert the new line and only does the insertion if the current line isn't already the line to be inserted. If the key line can appear at the end of the file then you'd need to test for key and add the new line in an END statement too: END{if (p~/key:\\/) print n}.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • Thank you very much. As you said, sed is just a tool for a single line operation, is it possible to write in awk that can prevent it from adding a line again on consecutive runs? Many thanks! – crackpot Apr 17 '16 at 03:36
  • 1
    Of course, that would be trivial and I've edited my answer to show it. I see you accepted a sed answer so I hope you're going to ask that poster for the same enhancement as I am looking forward to seeing how the sed script will be enhanced to handle this minor requirements update. Please leave a comment for me when the sed script is updated so I remember to come look at it, thanks. – Ed Morton Apr 17 '16 at 15:25
  • Thank you very much. awk seems complicated to me at first and I was asking a question of sed usage: hence, I thought sed answers was the way. I didn't know I could set multiple reply to be the answer. Sorry about that. i didn't know that awk may handle multiple lines until you told me so. That was inspiring! Thank you again! – crackpot Apr 18 '16 at 00:58
  • 1
    I highly recommend you read the book Effective Awk Programming, 4th Edition, by Arnold Robbins if you're going to be doing any more text processing and have awk available (e.g. on any UNIX installation). – Ed Morton Apr 18 '16 at 02:36
  • Thanks, I'll take a look! – crackpot Apr 19 '16 at 10:17