awk find and replace with regex and environment variable

Question

This is my text:

    set $domain_name some-not-important-name.com;

I want to reach this:

    set $domain_name test.com;

This is what I tried so far:

DOMAIN_NAME=test.com
awk '{ sub(/ set $domain_name(.*)/, " set $domain_name $DOMAIN_NAME;") }1' file

current result:

    set $domain_name some-not-important-name.com

Update 1

The reason I added multiple whitespaces is that, the line belongs to nginx config and it has multiple spaces.

Since there are both charset and set in my file and they're not the first character of the line, so I should replace set with space before.

What do you mean by "this is my text"? BTW, `DOMAIN_NAME` is not an environment variable, and parameter substitution does not occur between single quotes. — user1934428, Aug 17 '23 at 15:21
@user1934428 I mean this is the line I want to find and replace. Also, yes `DOMAIN_NAME` is a variable I set but is not an env-var. I also tried with double quotes but no luck. — Saeed, Aug 17 '23 at 15:25
I'm not familiar with `nginx` so .... do you need to maintain the *multiple whitespaces*? — markp-fuso, Aug 17 '23 at 15:26
@markp-fuso the `nginx` doesn't matter. Also it doesn't care if I have indentations or not, but for being more readable and having better blocks, I prefer to keep those spaces. — Saeed, Aug 17 '23 at 15:28
@Saeed : Probably you made a mistake with double-quoting. If you try something unsuccessfully, edit your question so that we can see your new attempt. — user1934428, Aug 18 '23 at 06:00
I'm surprised to see you trying to expand shell variables inside an awk script given all of the questions you've had answered and the fact you've done this the right way in some of your previous scripts (e.g. `-v host="${HOSTNAME,,}"` in https://stackoverflow.com/q/76851963/1745001). Anyway,- please read [how-do-i-use-shell-variables-in-an-awk-script](https://stackoverflow.com/questions/19075671/how-do-i-use-shell-variables-in-an-awk-script). — Ed Morton, Aug 18 '23 at 11:56
@EdMorton somewhere I need to expand them, because it's required to use them. Shouldn't I use them? I may not notice your previous answers and comments carefully. — Saeed, Aug 18 '23 at 13:51
Just read that article I referred you too, it explains everything. — Ed Morton, Aug 18 '23 at 13:56
@EdMorton thanks, I read it, but `echo ${HOSTNAME,,}` prints hostname all in lowercase. Do you mean use like this? `lowercase_hostname=${HOSTNAME,,}`, then `awk -v host="$(lowercase_hostname)" ...` I tried `ENVIRON[...]`, but it's not an env-var. It's a user-defined var. — Saeed, Aug 18 '23 at 16:10
I think you're overthinking this. `-v x="$y"` sets an awk variable named `x` to the contents of a shell variable named `y`. To convert the contents of a shell variable to lower case you use `"${y,,}"` instead of just `"$y"`. So if you want the awk variable `x` to contain the contents of the shell variable `y` after being converted to lower case then you use `-v x="${y,,}"`. So if you want an awk variable `dom` to contain the contents of the shell variable `DOMAIN_NAME` as-is, just write `-v dom="$DOMAIN_NAME"` — Ed Morton, Aug 18 '23 at 20:22
As that article I referenced shows you, if your shell variable isn't exported you can still access it through ENVIRON by puting it into the environment of the call to awk with (all on the same line) `y="$y" awk 'BEGIN{x=ENVIRON["y"]}'` — Ed Morton, Aug 18 '23 at 20:33

markp-fuso · Accepted Answer · 2023-08-17T16:02:59.343

UPDATE: (maintains spacing)

Setup:

$ cat file
ignore this line
    set    $domain_name    some-not-important-name.com;
ignore this line

One awk idea:

DOMAIN_NAME=test.com

awk -v dn="${DOMAIN_NAME}" '                                            # set awk variable "dn" to OS variable value
$1 == "set" && $2 == "$domain_name" { ndx = index($0,$3)                # find location of 3rd field
                                      $0  = substr($0,1,ndx-1) dn ";"   # rebuild current line
                                    }
1                                                                       # print all lines to stdout
' file

Assumptions:

there are only 3 white-spaced delimited fields in the line of interest
$3 does not match the strings set nor $domain_name
$3 does not contain white space

This generates:

ignore this line
    set   $domain_name    test.com;
ignore this line

Original answer: (does not maintain spacing)

Setup:

$ cat file
ignore this line
    set    $domain_name    some-not-important-name.com;
ignore this line

Assuming the line of interest has just the 3 space-delimited fields ...

One awk idea:

DOMAIN_NAME=test.com

awk -v dn="${DOMAIN_NAME}" '
$1 == "set" && $2 == "$domain_name" { $3 = dn ";"}   # redefine the 3rd field as dn + ";"
1                                                    # print all lines to stdout
' file

# or as a one-liner:

awk -v dn="${DOMAIN_NAME}" '$1 == "set" && $2 == "$domain_name" { $3 = dn ";"} 1' file

This generates:

ignore this line
set $domain_name test.com;
ignore this line

Thanks, no problem. I add spaces in my bash file for that for a workaround. — Saeed, Aug 17 '23 at 15:30
Yep, assignment causes awk to recalculate fields removing whitespace. Substitution is the key. — David C. Rankin, Aug 17 '23 at 22:52

score 3 · Answer 2 · answered Aug 18 '23 at 12:05

Using any POSIX awk:

$ awk -v dom="$DOMAIN_NAME" 'match($0,/^[[:space:]]*set[[:space:]]+\$domain_name[[:space:]]+/) { $0=substr($0,1,RLENGTH) dom ";"} 1' file
    set $domain_name test.com;

That will work and retain spacing no matter which characters/strings are present in the input line, including regexp metachars, backreferences, substrings of earlier fields, white space, or anything else.

score 0 · Answer 3 · answered Aug 17 '23 at 15:31

0

This would work

v=' set $domain_name some-not-important-name.com;'
awk -v d=test.com '{ sub(/ set \$domain_name .*/, " set $domain_name " d ";")}1' <<<"$v"

you have to escape the $

answered Aug 17 '23 at 15:31

Diego Torres Milano

65,697
9
111
134

Thanks, but the variable `d` is not static in this case. It's a user-defined variable that may change in the future. – Saeed Aug 17 '23 at 20:04

awk find and replace with regex and environment variable

3 Answers3