Replace a string in bash, with a variable based on that string

Question

I am trying to parse a file for IP addresses and then replace them with a 1 for 1 mapped IP address. For example, find all instances of 1.1.1.1 and replace these with 100.100.1.1, and all instances of 5.5.5.5 and replace those with 100.100.5.5. There may be thousands of IPs, so I do not want to parse the file for each substitution using static sed commands.

I am trying to use an associative array for this, but I am falling short.

echo "1.1.1.1 5.5.5.5" > testfile.txt
declare -A testarray
testarray[ip_1.1.1.1]="100.100.1.1"
testarray[ip_5.5.5.5]="100.100.5.5"
sed -r "s/(([0-9]{1,3}\.){3}[0-9]{1,3})/${testarray[ip_\1]}/g" testfile.txt

Using the capture group inside the variable doesn't work. However using the text directly does.

sed -r "s/(([0-9]{1,3}\.){3}[0-9]{1,3})/${testarray[ip_1.1.1.1]}/g" testfile.txt
100.100.1.1 100.100.1.1

Of course this replaces all IPs with 100.100.1.1, I want to replace based on the array association.

Is there any way to take a capture group and use it in a variable for a replacement in bash? I have tried a few variants with quotes as well as building a variable in advance that still depends on '\1' but it either fails, or just returns the literal output of the variable. I don't know if there is a better tool than sed for this. I also tried using variables instead of an array, but I ran into the same issue.

You are calling the index dinamically with `ip_\1`. That is, using the captured group. This is not possible as you are doing it now, but probably need to use `sed` `e` flag to execute stuff. See for example [this answer by anubhava](http://stackoverflow.com/a/34080390/1983854) for a nice example. — fedorqui, May 17 '16 at 13:34
Perl supports both substitution and associative arrays (hashes). I'd use it instead of sed. — choroba, May 17 '16 at 13:43
Thanks for the link @jil, I have not managed to get the linked awk solution to work yet, but I have no experience with Awk so I can't easily tell if this solves the problem. From my reading of it, it may not. — suretass, May 21 '16 at 11:32
Don't forget you can use your array to create multiple expressions with the `-e` option with `sed` and change as many or as few IP-pairs as you would like. If the file has 1000 lines, you could conceivably do it in single call. — David C. Rankin, May 24 '16 at 07:10
[Replace a field with values specified in another file](http://stackoverflow.com/questions/12400217/replace-a-field-with-values-specified-in-another-file) has `sed`, `awk`, & `perl` methods to do this. — agc, May 24 '16 at 20:52

score 0 · Answer 1 · answered May 24 '16 at 20:51

POSIX shell script, input is testfile.txt, comma seperated "from to" pairs are in replacements.txt:

echo 1.1.1.1 100.100.1.1 ,5.5.5.5 100.100.5.5 | tr ',' '\n' > replacements.txt
echo "1.1.1.1 6.7.8.9 5.5.5.5" | tr ' ' '\n' > testfile.txt

while read x ; do
    while read a b ; do
        if [ $a = $x ] ; then
            x=$b
            break
        fi
    done < replacements.txt
    echo $x
done < testfile.txt

Output:

100.100.1.1
6.7.8.9
100.100.5.5

Notes: inefficient, it rereads at least one line of replacements.txt for every line of testfile.txt, but that does permit replacement lists bigger than bash's available memory.

Replace a string in bash, with a variable based on that string

1 Answers1