2

hi trying to replace the following string with a long one :

@x@ 

with string that I got from the command line:

read test    
sed -i --backup 's/@x@/'${test}'/g' file.json README.md

but it is working only for 1 word, it is not working if there is space between word . even between quotes

sed: 1: "s/@x@/string test string: unterminated substitute in regular expression
Cyrus
  • 84,225
  • 14
  • 89
  • 153
Tuz
  • 1,810
  • 5
  • 29
  • 58
  • 4
    Variable references should always be in double-quotes to avoid problems like this. Use something like `sed -i --backup 's/@x@/'"${test}"'/g' file.json README.md` or just `sed -i --backup "s/@x@/${test}/g" file.json README.md`. [Shellcheck.net](https://www.shellcheck.net) is good at spotting common problems like this. – Gordon Davisson Aug 26 '18 at 07:46
  • Also, you should add some sanity checking so that the embedded variable does not include characters that will be interpreted as part of the sed command. For example, `${test//\//}` will strip out forward slashes using Parameter Expansion. – ghoti Aug 30 '18 at 16:03

3 Answers3

6

if case you run it on MacOS and struggling with "unterminated substitute in regular expression", there is an easier explanation for this:

MacOS has slightly other version of sed than usually is on linux. -i requires a parameter. If you have none, just add "" after -i

sed -i "" --backup 's/@x@/'${test}'/g' file.json README.md

or for example if you just have to delete dome line, this works on linux, but brings “invalid command code” on MacOS

sed -i 39d filenamehere.log

and this works on MacOS

sed -i "" 39d filenamehere.log
python_kaa
  • 1,034
  • 13
  • 27
1

The problem originates from the way you are using the single-quotes. Currently you are terminating your input behind the 2. single-quote. See the Error message, it makes you aware of the fact that it is missing something.

If you have a file with the following content:

foo @x@ foo

Than you can replace the content e.g. with the following command:

sed 's/@x@/bar foo bar/' foo.txt > foo2.txt

And get:

foo bar foo bar foo

If you need to pass in a variable the comment from Gordon Davisson shows you the right way.

By the way, if you want to use the inplace option, on my linux you would need to use the command like this:

sed -i.old "s/@x@/${test}/" foo.txt

But I think this might depends on your enviroment (mac?).

morecore
  • 900
  • 3
  • 14
  • 32
  • But i do not send string with quotes . – Tuz Aug 26 '18 at 07:45
  • Quotes aren't the only potential problem, so are `/`s, `&`s, `\`s, and many more. Once you solve the quotes problem you still have the problem that you cannot robustly replace a regexp with just any string using sed. See https://stackoverflow.com/q/29613304/1745001 for what's required to pre-process a replacement string. – Ed Morton Aug 26 '18 at 11:39
  • wrt `The single-quotes ... needs to be escaped or removed.` - no they don't. You can't escape single quotes within a single-quote delimited string/script, and it's correct to enclose a script in single quotes but open up a section to let the shell interpret it, e.g. to expand a variable. The OPs quoting mistake was just not double-quoting the variable - `'s/@x@/'"${test}"'/g'`. There are other issues of course as I mentioned in my other comment and answer. Also, wrt UNIX variants - AFAIK macs run the BSD UNIX variant, not the linux UNIX variant. – Ed Morton Aug 26 '18 at 14:04
0

sed doesn't understand strings where a string is a series of literal characters. It replaces a regexp (not a string) with a backreference-enabled "string" (also not a string) all within a set of delimiters (which ALSO require careful handling in both the regexp and the replacement). See Is it possible to escape regex metacharacters reliably with sed for more info.

To replace a string with another string the simplest approach is to just use a tool that understands strings such as awk:

$ cat file
before stuff
foo @x@ bar
after stuff

$ cat tst.awk
BEGIN {
    old = ARGV[1]
    new = ARGV[2]
    ARGV[1] = ARGV[2] = ""
}
s = index($0,old) { $0 = substr($0,1,s-1) new substr($0,s+length(old)) }
{ print }

$ test='a/\t/&"b'

$ awk -f tst.awk '@x@' "$test" file
before stuff
foo a/\t/&"b bar
after stuff

The above will work no matter what characters test contains, even newlines:

$ test='contains a
newline'
$ awk -f tst.awk '@x@' "$test" file
before stuff
foo contains a
newline bar
after stuff
Ed Morton
  • 188,023
  • 17
  • 78
  • 185