1

I am trying to do something that should be simple and routine. I want to search the file /etc/dhcpcd.conf and change an uncommented static ip address assigned to eth0. The sample test file is shown here:

interface eth0
# test variants of comments
#static ip_address=192.168.21.40/24
# static ip_address=192.168.21.40/24
 #static ip_address=192.168.21.40/24
 # static ip_address=192.168.21.40/24

#the line to match and change
static ip_address=123.223.122.005/24

I attempted to implement a solution here: here’s a much-less-cryptic way of doing it in awk answered by Scott - Слава Україні

I chose the AWK solution because it is easy to read however, I have not been able to modify the answer to work for me. I haven't used sed or awk before so I wrote a test script.

#!/bin/bash
# testing on a copy of the config file
conf_file="/home/user/dhcpcd.conf"
# BASH variable with fake test ip
new_ip=111.222.123.234

cat "$conf_file" |  awk $'
   /interface /                    { matched=0 }
   /interface eth0/                { matched=1 }
    matched                        { gsub( "^\\s*static ip_address=•*" , "static ip_address=111.222.123.234" ) }
                                   { print }
   '

This AWK script is to find the eth0 stanza, then find static ip_address= any text. and replace this with static ip_address=111.222.123.234. The exception is for # commented lines. For testing I have hard coded the test new ip into the AWK script. I want to use the variable $new_ip.

Initially had a problem with the gsub function. The •* intended to match any text caused issues because I began using .*. The manual says gsub uses the character and not a full-stop. without actually highlighting the need to use Alt-7. That took me a while to find but that problem is now solved.

The 1st remaining problem I now have is that the new ip is inserted in the line rather that replacing the old ip. The output is: static ip_address=111.222.123.234123.223.122.005/24 So the regex is, or should be, matching the whole line, including the old ip to be replaced, but it isn't.

The 2nd problem I have is that I need awk to use the bash variable new_ip. I have looked and searched but I cannot find an answer that works.

I think this should work:

cat /home/mimir/dhcpcd.conf |  awk -v sip=$new_ip  $'
   /interface /                    { matched=0 }
   /interface eth0/                { matched=1 }
    matched                        { gsub( "^[:blank:]*static ip_address=•*" , "static ip_address=sip" ) }
                                   { print }

 '

but it doesn't. The output was

static ip_address=sip123.223.122.005/24 so awk didn't recognise sip as a variable. I am doing something wrong, but I can't see it. It shouldn't be so difficult to do something so simple.

Any help would be much appreciated.

EDIT No.2: For the benefit of those landing here looking for a solution, this is my working test script:

 #!/bin/bash
#  Test reading and setting of the ip address in /etc/dhcpcd.conf  using AWK

# source /etc/dhcpcd.conf  which always includes the lines:
# interface eth0
#  static ip_address=www.xxx.yyy.zzz
# copy of config file for testing
cfg_file="/home/user/dhcpcd.conf"
# fake test ip
new_ip="114.113.112.111"

awk -i inplace -v sip=$new_ip  $'
   /interface /                    { matched=0 }
   /interface eth0/                { matched=1 }
    matched                        { gsub( "^[[:blank:]]*static ip_address=.*" , "static ip_address="sip ) }
                                   { print }
   ' $cfg_file
exit

This is now working because of a combination of advice received below in the comments. This is an answer, but it is not my answer. It works but is not perfect.

This has some limitations. For a technically better answer, see Ed Morton's answer below.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
dazz
  • 119
  • 2
  • 14
  • 1
    You can start by removing the `cat` and pipe `|`. e.g. `awk ... "$conf_file"` – Jetchisel May 06 '23 at 00:57
  • try `[[:blank:]]` . I don't have time to debug this now. Good luck. – shellter May 06 '23 at 01:00
  • `new_ip=111.222.123.234; conf_file="/home/user/dhcpcd.conf"; printf '%s\n' "/interface eth0/;/^static ip_address/,s/^\([^=]*\).*/\1=$new_ip/" ,p Q | ed -s dhcpd.conf` – Jetchisel May 06 '23 at 01:10
  • 3
    please give link to: "The manual says gsub uses the character • and not a full-stop." – jhnc May 06 '23 at 02:13
  • @jhnc http://www.endmemo.com/r/gsub.php shows • (Alt-7), not fullstop. – dazz May 06 '23 at 02:42
  • 2
    that is documentation for `R` not `awk` – jhnc May 06 '23 at 02:45
  • @Jetchisel The first sentence of this link https://www.oreilly.com/library/view/effective-awk-programming/9781491904930/ch04.html says `In the typical awk program, awk reads all input either from the standard input (by default, this is the keyboard, but often it is a pipe from another command) or from files whose names you specify on the awk command line. ` Which gave me 2 options. Piping I know so that's what I went for. I am finding awk and gsub to be on a steep learning curve. Easy when you know how. Not so easy to learn. – dazz May 06 '23 at 02:52
  • @jhnc That came to the top when I was searching gsub. I don't know what "R" is. I will try again with a full stop. – dazz May 06 '23 at 02:54
  • Anyway, it looks like a typesetting mistake on that page. I'm fairly certain R uses period (`.`) also – jhnc May 06 '23 at 02:57
  • OK so I have tried `*.[[:blank:]]*` and replaced `•` with a full stop `.` Now I get: `static ip_address=sip` so the variable declaration isn't being recognised by awk. I am still piping the file into awk `cat | awk .....` – dazz May 06 '23 at 03:01
  • awk doesn't expand variables in strings: `"static ip_address=sip"` ; you catenate: `"static ip_address=" sip` – jhnc May 06 '23 at 03:03
  • 1
    @dazz, yes but stop doing [UUOC](https://mywiki.wooledge.org/BashFAQ/119) – Jetchisel May 06 '23 at 03:07
  • @jhnc Bingo. That gives the right answer. Thanks. – dazz May 06 '23 at 03:07
  • OK so if piping in a file is not good, then I need to use the alternative. Google found this https://stackoverflow.com/questions/8019617/how-to-write-finding-output-to-same-file-using-awk-command The answer by kernorb looks promising `$ gawk -i inplace '{ gsub(/foo/, "bar") }; { print }' file1 file2 file3` but I don't know if using `gawk` will require changes to my awk statements in some subtle way. – dazz May 06 '23 at 03:21
  • @jhnc UUOC LOL. Guilty as charged ☺ – dazz May 06 '23 at 03:50
  • 1
    Many tools/languages have commands named `gsub()` but they aren't all the same, don't assume if google finds information about `gsub()` that it's about the `gsub()` that's part of `awk`, e.g. https://apidock.com/ruby/String/gsub is about the `gsub()` that's part of `Ruby`, https://metacpan.org/pod/String::Gsub is `perl`, and you already found the one in `R`. – Ed Morton May 06 '23 at 11:32
  • 1
    That last question you referenced in [a comment](https://stackoverflow.com/questions/76186654/using-awk-gsub-to-find-replace-a-setting-in-a-config-file/76186870#comment134356524_76186654) is about writing output back to the input file - that's trivial and something you can do with any tool once you're getting the output you want. You should focus on getting the output you want and THEN tweaking it to overwrite the input. – Ed Morton May 06 '23 at 11:38
  • 2
    Regarding the addition to your question where you say "this is my working test script:" - in addition to the UUOC, that script would fail if `interface` existed in a comment or if `new_ip` somehow ended up containing `&`, or if the white space between strings wasn't always a single blank, or if `new_ip` contained a space or a globbing char (depending on environment settings), and it has an undesirable `$` before the awk script that's asking the shell to expand escape sequences, and it's unnecessarily testing for `interface` twice - the answer I provided avoids all those issues. – Ed Morton May 06 '23 at 11:47
  • 1
    @Ed Morton I thought I would have a go at adding some regex to my test script to address the deficiencies identified by you. It quickly became apparent that the regex was going to be an unreadable, cryptic and unmaintainable mash which only highlights the elegant simplicity of your answer. – dazz May 08 '23 at 00:34
  • [Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.](https://blog.codinghorror.com/regular-expressions-now-you-have-two-problems/) :-). Regexps are, of course, invaluable, just keep them simple and don't rely on them more often than necessary. – Ed Morton May 08 '23 at 02:32
  • @EdMorton I tried replicating your code to change ip, route and dns name server and hit an unexpected problem. I have added Edit No.3 to my original post to explain the problem in detail. I get a digit concatenated to the beginning of the ip addresses. I can't see anything in the awk code that might cause that. If you could take a look, that would be appreciated. – dazz May 10 '23 at 00:18
  • [chameleon questions](https://meta.stackexchange.com/questions/43478/exit-strategies-for-chameleon-questions) are strongly discouraged on this forum. Please put this question back as it was when you got answers and then ask a new question if you have one. – Ed Morton May 10 '23 at 10:35
  • Having said that - any time any text starts showing up in unexpected places the first thing you should check for is DOS line endings in your input files and your script. See [why-does-my-tool-output-overwrite-itself-and-how-do-i-fix-it](https://stackoverflow.com/questions/45772525/why-does-my-tool-output-overwrite-itself-and-how-do-i-fix-it) and if it's not that then ask a question about it. – Ed Morton May 10 '23 at 10:45
  • @EdMorton OK I read the chameleon question link but I don't think that has happened here. The problem I have, as stated in the first para of my original post, remains extant. The scope of the question remains unchanged. I read the dos line ending link. All of the relevant files have been created, copied or edited on the CLI so low risk of a DOS file messing things up. I did run dos2unix on each file, then checked them with cat -vE to be sure. I tried rebooting the OS just in case. The test code just changes one ip, to simplify as much as possible. – dazz May 11 '23 at 08:17
  • I took another look at my script and noticed it was leaving behind a digit from the start of the original IP address. That has nothing to do with all the extra code and complexity you added to your question doing a bunch of other things in shell and calling the script multiple times, it was just a plain old bug in my answer to your original question, reproducible when calling the script once standalone, so I rolled your question back to what it was when you got answers and fixed my script. – Ed Morton May 11 '23 at 11:44
  • @EdMorton OK. Thanks for your input here. There is no realistic possibility I would have ended up with a script that looks like yours (with or without a bug). If it wasn't for people volunteering their time and effort to answer questions on these forums, things would be much more difficult for people like me. I did try running the test script on another machine to see if the problem was some issue with my OS setup. It didn't run on the 2nd machine because the 'inplace' option sprung an error. I needed to install 'gawk' and then it ran OK. – dazz May 12 '23 at 08:35

1 Answers1

5

Using any POSIX awk:

$ cat tst.sh
#!/usr/bin/env bash

new_ip=111.222.123.234

awk -v sip="$new_ip" '
    $1 == "interface" {
        gotIface = ($2 == "eth0" ? 1 : 0)
    }
    gotIface && match($0,/^[[:space:]]*static[[:space:]]+ip_address=/) {
        $0 = substr($0,1,RSTART+RLENGTH-1) sip
    }
    { print }
' file

$ ./tst.sh
interface eth0
# test variants of comments
#static ip_address=192.168.21.40/24
# static ip_address=192.168.21.40/24
 #static ip_address=192.168.21.40/24
 # static ip_address=192.168.21.40/24

#the line to match and change
static ip_address=111.222.123.234

Regarding:

it needs to read and write to the same file

GNU awk has -i inplace for that (but it's still using a temp file behind the scenes), but for any Unix tool, tool, you can just do:

tmp=$(mktemp) &&
tool file > "$tmp" &&
mv -- "$tmp" file

e.g.

tmp=$(mktemp) &&
awk '{print FILENAME, $0}' file > "$tmp" &&
mv -- "$tmp" file

would prepend each input line with the file name and write the result back to the input file.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • Hi. Thanks for your answer to this question. I am sure it is right, but as a casual amateur Linux programmer with no prior knowledge or need for awk or gsub, your answer is difficult for me to follow. I could work my way through it and figure it out but if/when I came back to it in a year or three, it would be equally cryptic to me. Given my limited knowledge and capability, I put priority on readability/maintainability. This comment is not a criticism of your answer, it is my recognition of my limitations. – dazz May 06 '23 at 03:44
  • 1
    @dazz `awk` is a bit cryptic at first, but simple. It reads records (lines) from a file and applies the rules you write (the commands between `{ ... }`), one-at-a-time, in order, to each record. Above there are 2-rules. The first is triggered by `$1 == "interface"`, e.g. the first field in the line is `"interface"` and if so sets the variable `gotinterface` to the result of the *ternary* (shorthand `if/else`, if 2nd field is `"eth0"` then `gotinterface = 1`, otherwise `0`) Second rule search whole record for `"static ip_address="` and if so sets the record to the substring specified. – David C. Rankin May 06 '23 at 07:39
  • 1
    See [GNU Awk String Functions](https://www.gnu.org/software/gawk/manual/html_node/String-Functions.html#String-Functions) for specific details on `match` and the built-in variables `RSTART` and `RLENGTH` and then you will understand how the substring is obtained. – David C. Rankin May 06 '23 at 07:41
  • I wrote my first program about 45 years ago and over the years I have used a dozen or more languages. I like a lot of things about Linux but the cryptic nature of much of the syntax is not one of them. I have already spent far too much time troubleshooting my test script, and if it wasn't for the help received here, I would still be struggling. – dazz May 06 '23 at 09:59
  • 1
    @dazz my experience is similar to yours. awk is just a tiny subset of C wrapped in an implicit while-read loop that splits each input line into fields, with syntax that's a series of `condition { action }` statements. It's designed for text-processing and so does a lot of the common things required for text processing for you so you don't have to write all of the code to do it. You do still have to learn the basics of what it does to be able to use/understand it but the good news is that's a handful of well-documented pages of text (Effective AWK Programming, 5th Edition, by Arnold Robbins). – Ed Morton May 06 '23 at 11:13
  • Please feel free to ask questions if you find any part of the script in my answer confusing. We can't explain awk fundamentals in every answer but I'm happy to answer any specific questions. – Ed Morton May 06 '23 at 11:27
  • @Ed C is one of the languages I haven't used, although I have used close relatives. The file line splitting, condition { action } structure is what lured me to awk. Where I hit the rocks was the subtle details. In my specific case, I alone have total control of the config files, so I can live with (avoid) the failure cases you outlined. It is good enough. In the general case your more sophisticated solution would be the better choice. – dazz May 07 '23 at 04:55
  • @Ed I found the handful of well documented pages of text by Robbins here: https://www.gnu.org/software/gawk/manual/gawk.pdf Only 579 pages so yeah nah. I won't be reading that. I will bookmark it so I can find it if I need it. It never came up in any of my searches. – dazz May 07 '23 at 08:11
  • 1
    You don't need to read the whole book to understand awk enough to get going, just the first few pages. I always refer people to buy the book rather than just read the online reference material that the author also graciously provides for us as sales of the book are the only compensation the author gets for writing all of that documentation and providing/supporting GNU awk. – Ed Morton May 07 '23 at 13:38
  • 1
    I have tested the answer and confirm that it works. Thanks. – dazz May 12 '23 at 09:17