sed command to find and replace string with special characters to a string in command line

Question

I am trying to use sed to find/replace a string with special characters in a .property file.

This is my original line in the file:

kylin.source.hive.beeline-params=-n root --hiveconf hive.security.authorization.sqlstd.confwhitelist.append='mapreduce.job.*|dfs.*' -u jdbc:hive2://localhost:xxxx

I need to replace :

root --hiveconf hive.security.authorization.sqlstd.confwhitelist.append='mapreduce.job.*|dfs.*'

with

hadoop

and other string :

localhost

with

ip-00-00-00-000.ec2.internal

Final output needs to look like :

kylin.source.hive.beeline-params=-n hadoop -u jdbc:hive2://ip-00-00-00-000.ec2.internal:xxxx

I have tried a few different formats using sed:

sudo sed -i 's/root --hiveconf hive\.security\.authorization\.sqlstd\.confwhitelist\.append=\'mapreduce\.job\.\*\|dfs\.\*\'/hadoop' /usr/local/kylin/kylin.properties

sudo sed -i 's/root \-\-hiveconf hive\.security\.authorization\.sqlstd\.confwhitelist\.append\=\'mapreduce\.job\.\*\|dfs\.\*\'/hadoop' /usr/local/kylin/kylin.properties

sudo sed -r 's/root \-\-hiveconf hive\.security\.authorization\.sqlstd\.confwhitelist\.append\=\'mapreduce\.job\.\*\|dfs\.\*\'/hadoop' /usr/local/kylin/kylin.properties

I am not getting any output when I execute the above commands,its is waiting for another input.Can someone help me with this problem?

One issue I see is that you escape too much (not too little, as you seem to worry about). Hyphen is not a metacharacter except in character classes. In basic regular expressions (which is what `sed` uses unless you use the `-E` or `-r` flag) alternation is not supported, so the pipe character should *not* be escaped. You can't just randomly try and hope to get lucky; you need to choose a documentation source and consult it to see exactly what does and what does not need to be escaped. — , May 11 '20 at 06:41

Ron · Accepted Answer · 2020-05-11T17:13:24.543

0

Here:

sed -i "s#root.*dfs.*'#hadoop#g; s#localhost#ip-00-00-00-000.ec2.internal#g" /usr/local/kylin/kylin.properties

will output:

kylin.source.hive.beeline-params=-n hadoop -u jdbc:hive2://ip-00-00-00-000.ec2.internal:xxxx

or to replace only first occurrence in file:

sed -i "0,/root.*dfs/{s#root.*dfs.*'#hadoop#g; s#localhost#ip-00-00-00-000.ec2.internal#g}" /usr/local/kylin/kylin.properties

edited May 11 '20 at 17:13

answered May 10 '20 at 23:40

Ron

5,900
2
20
30

Thanks,I have tried with the command, I need the sed command to replace only the first occurrence in a file. I tried omit the g flag but this is only helpful to the first instance of the search string in each line will be replaced but not the only first occurence of the entire file. I have not used sed search and replace much previously.Can you please let me know How could I do that? – Sai May 11 '20 at 13:55

Ed Morton · Answer 2 · 2020-05-11T13:22:58.597

When you want to replace a literal string with a literal string, use a tool like awk that understands literal strings, not a tool like sed that only understands regexps and backreference-enabled replacements which you then need to carefully escape the metacharacters and delimiters of in different ways (see is-it-possible-to-escape-regex-metacharacters-reliably-with-sed:

$ cat tst.awk
BEGIN {
    map["root --hiveconf hive.security.authorization.sqlstd.confwhitelist.append='mapreduce.job.*|dfs.*'"] = "hadoop"
    map["localhost"] = "ip-00-00-00-000.ec2.internal"
}
{
    for (str in map) {
        if ( s = index($0,str) ) {
            $0 = substr($0,1,s-1) map[str] substr($0,s+length(str))
        }
    }
    print
}

$ awk -f tst.awk file
kylin.source.hive.beeline-params=-n hadoop -u jdbc:hive2://ip-00-00-00-000.ec2.internal:xxxx

sed command to find and replace string with special characters to a string in command line

2 Answers2