sed with su misinterprets sed negation

Question

This is simplified version of my actual problem. Here's my test.data file

test.data
=========
PROD SEARCH_URL = "google.com" db name "customers"
TEST SEARCH_URL = "google.com" db name "emp"

For all the lines containing "TEST", I want to change the search domain from "google.com" to "bing.com"

Below sed command works fine to achieve this:

sed -rin '/PROD/!s/bing.com/google.com/' test.data

But the way my test environment is set up, I need to run the sed command using "su" as:

su root "sed -rin '/PROD/!s/bing.com/google.com/' test.data"

And this results in error

-bash: !s/bing.com/google.com/': event not found

From other stackoverflow quesrions, I have tried: escaping ! and enclosing it in ''. Nothing seems to work

Have also tried "set +H" as below and it doesn't work:

su root "set +H && sed -rin '/PROD/!s/bing.com/google.com/' test.data"

What am I missing?

This may be a result of your simplification but surely the requirement "for all the lines containing TEST" would be better met with `/TEST/s/...` rather than `/PROD/!s/...`? And I think your `bing` and `google` may be the wrong way around. — paxdiablo, Sep 28 '18 at 00:32
That is correct, I probably oversimplified. the lines are much more complex so I kind of need to use the ! operator to exclude the lines that I don't want. But if you run my example commands, its easy to reproduce the problem — Amol, Sep 28 '18 at 00:36
No probs, just thought I'd clarify. FWIW, the "event" reminds me of the csh history feature where `!` would be used to recall a specific command. I *think* this functionality is also in `bash` but I'm not sure why it would be relevant given it's inside single quotes. — paxdiablo, Sep 28 '18 at 00:38
you should escape `!`, and use `-c` option when running command with `su`. such as `su root -c "sed -rin '/PROD/\!s/bing.com/google.com/' test.data"` — Feng, Sep 28 '18 at 03:05
@Feng, I have already tried escaping the negation to \! and I am actually running the command with -c, that does not make any difference! — Amol, Sep 28 '18 at 03:37
@Amol I tested the two commands, they work well. `su root -c "sed -r '/TEST/s/google.com/bing.com/' test.data"` and `su root -c "sed -r '/PROD/\!s/google.com/bing.com/' test.data"` — Feng, Sep 28 '18 at 06:13
The backslash escaping should work. Also, depending on your bash version you may have a slightly different behavior, e.g. https://stackoverflow.com/a/41273889 — vdavid, Sep 28 '18 at 07:11

paxdiablo · Answer 1 · 2018-09-28T01:35:05.680

It appears the intepretation of the ! history character is actually being done by the current shell which means turning it off within the su-created shell is too late.

Further, the line:

set +H && su ...

won't help since it looks like the entire line is subjest to history modifications as soon as you hit ENTER, before the set +H takes effect. Even set +H ; su ... doesn't seem to help with that.

However, the following does appear to work, turning off history before the line that would use it:

set +H
su ...
set -H

See the following transcript for detail:

pax$ cat test.data
PROD google.com
ABCD google.com
DEFG google.com

pax$ set +H
pax$ su root -c "sed -rin '/PROD/!s/google/bing/' test.data"
Password: <blah blah blah>

pax$ cat test.data
PROD google.com
ABCD bing.com
DEFG bing.com

There is also a way to do this with a tool that doesn't require the ! character at all (if, for some reason, the above does not work for you).

As per the following transcript, awk will do the job nicely (you just have to emulate the sed in-place editing:

pax$ cat test.data
PROD google.com
ABCD google.com
DEFG google.com

pax$ su root -c "awk '/PROD/{gsub(/google/,\"bing\")}1' test.data >xx && mv xx test.data"
Password: <blah blah blah>

pax$ cat test.data
PROD google.com
ABCD bing.com
DEFG bing.com

You have to escape the " characters within the string due to the outer " quotes but that's about as simple as it can be made, while still honoring the original intent (as I understand it).

So all these commands are invoked remotely over a transient ssh connection. Which means `set +H` and `sed` need to be run as part of a single command invocation if at all :( — Amol, Sep 28 '18 at 14:56

sed with su misinterprets sed negation

1 Answers1