string replacement using awk command

Question

This is a continuation to my previous post string replacement using awk command which is not 100% fully resolved.. To make it clearer..

(The source file and code I am using are at the bottom of the post)

requirement 1

If I dont have a lookup data(Eg, for "u_no" is not in the reference file, the current solution prints an empty field.

aaaa
uid=1a1a
pwd=1b1b
u_no=

I want that to pick the value from the source file instead..

aaaa
uid=1a1a
pwd=1b1b
u_no=12345

requirement 2

If the data in the source file or reference file is mixed up, it should still do the replacement properly.

That is, when the ref file is changed like this:

block,parameter,value
aaaa,uid,1a1a
aaaa,pwd,1b1b
bbbb,uid,2a2a
zzzz,pwd,9b9b
zzzz,uid,9a9a
bbbb,pwd,2b2b

For this, I am getting

aaaa
uid=1a1a
pwd=1b1b
u_no=
bbbb
uid=2a2a
pwd=9b9b
zzzz
uid=9a9a
pwd=2b2b

** I must get**

aaaa
uid=1a1a
pwd=1b1b
u_no=12345    #points to my req 1 
bbbb
uid=2a2a
pwd=2b2b
zzzz
uid=9a9a
pwd=9b9b

I want the search and replace to happen based on the blocks aaaa,bbbb,zzzz,etc.. Not sequentially. (For eg, here the pwd value for bbbb should be 2b2b and not 9b9b).

source file(src.txt)

aaaa
uid=xxxx
pwd=nnnn
u_no=12345
bbbb
uid=yyyy
pwd=eeee
zzzz
uid=yyyy
pwd=eeee

Note uid and pwd value for bbbb is same as that of zzzz in src file.

reference file(ref.txt)

block,parameter,value
aaaa,uid,1a1a
aaaa,pwd,1b1b
bbbb,uid,2a2a
zzzz,pwd,9b9b
zzzz,uid,9a9a
bbbb,pwd,2b2b

Note: The order of records in the ref file might also change as I mentioned in requirement 2. The lookup value for uid and pwd for bbbb is different than that of zzzz. But I need the result as follows:-

Required Output file(src_out.txt)(including the requirements 1 and 2)

aaaa
uid=1a1a
pwd=1b1b
u_no=12345
bbbb
uid=2a2a
pwd=2b2b
zzzz
uid=9a9a
pwd=9b9b

Code

$ cat tst.awk
BEGIN { FS = "[,=]+" }
NR==FNR {
    if (NR>1) {
        map[$1,$2] = $3
    }
    next
}
{
    if (NF==1) {
        key = $0
    }
    else {
        $0 = $1 "=" map[key,tolower($1)]
    }
    print
}

$ awk -f tst.awk ref.txt src.txt

This works fine for my initial expectation, but the requirement 1 and requirement 2 which I specified here are now added and so I need the solution for these.

Edit your question to make it MUCH clearer and more explicit what the difference is between this and your previous questions (http://stackoverflow.com/q/28963149/1745001 and http://stackoverflow.com/q/28849774/1745001) as I for one can't see any difference at all, and also tell us in what way(s) the previous solution (http://stackoverflow.com/a/28965212/1745001) does not work for your new requirements (whatever they are) - show the output it produces for comparison with the output you want. You need to put more thought and effort into asking the question so we can help you. — Ed Morton, Mar 12 '15 at 14:58
possible duplicate of [Search replace string in a file based on column in other file](http://stackoverflow.com/questions/20327179/search-replace-string-in-a-file-based-on-column-in-other-file) — NeronLeVelu, Mar 13 '15 at 06:59
@EdMorton -- Edited the post....Explained it as much as I can! ... Please have a look now... Thanks for your patience...! :-) — Sanish, Mar 13 '15 at 08:44
@Sanish yes, now it is clear so I've posted an updated solution to only use the ref data if it exists. As I've said repeatedly, the original script is completely independent of the order of lines in the ref file so idk why you keep bringing that up. The script I provided simply CANNOT do what you say it does above and mix up the values given the input files you show - all it does is map keys to values, no magic and no order implied. THINK about it and insert print statements if it's not clear. — Ed Morton, Mar 13 '15 at 12:48

Jotne · Accepted Answer · 2015-03-13T10:00:04.660

0

Here is one solution with awk

awk -F= '
FNR==NR {
    split($0,b,",")
    a[b[1] FS b[2]]=b[3]
    next} 
!/=/ {
    f=$1
    print
    next} 
    {
    print $1"="(a[f FS $1]?a[f FS $1]:$2)}
' ref.txt src.txt
aaaa
uid=1a1a
pwd=1b1b
u_no=12345
bbbb
uid=2a2a
pwd=2b2b
zzzz
uid=9a9a
pwd=9b9b

edited Mar 13 '15 at 10:00

answered Mar 13 '15 at 09:49

Jotne

40,548
12
51
55

It works fine... If in case I need to send parameters into the AWK program.. how to send here? ... here I want to take 3rd column or 4th column or 5th column from the reference file based on certain conditions.... so the key would become either... a[b[1] FS b[2]]=b[3].....or.... a[b[1] FS b[2]]=b[4].....or.....a[b[1] FS b[2]]=b[5]...while reading the reference file.... how to parameterize this b[...]... ??? – Sanish Mar 14 '15 at 14:16
@Sanish Read my answer regarding getting data inn to `awk` here: http://stackoverflow.com/questions/19075671/how-to-use-shell-variables-in-awk-script – Jotne Mar 14 '15 at 14:58
Sir... It helped a lot... thanks... I am using Sun OS 5.10... It is showing a syntax error for the line in your code...."!/=/ {" .... is there a workaround for this? ..... but it worked pretty well in red hat.... – Sanish Mar 16 '15 at 14:22
This `!/=/ {` test if the line does not contain `=` and should work on all version of `awk`. You can try to replace it with `$0!~/=/ {` – Jotne Mar 17 '15 at 06:29
@Sanish Use nawk (new awk) or /usr/xpg4/bin/awk (POSIX awk) with Solaris. – Jotne Mar 17 '15 at 07:40

score 0 · Answer 2 · answered Mar 13 '15 at 12:42

0

Tweaked to test for the presence of the src values in the ref file:

$ cat tst.awk
BEGIN { FS = "[,=]+" }
NR==FNR {
    if (NR>1) {
        map[$1,$2] = $3
    }
    next
}
{
    if (NF==1) {
        key = $0
    }
    else if ( (key,tolower($1)) in map ) {
        $0 = $1 "=" map[key,tolower($1)]
    }
    print
}

$ awk -f tst.awk ref.txt src.txt
aaaa
uid=1a1a
pwd=1b1b
u_no=12345
bbbb
uid=2a2a
pwd=2b2b
zzzz
uid=9a9a
pwd=9b9b

answered Mar 13 '15 at 12:42

Ed Morton

188,023
17
78
185

Will check and let you know sir... Thanks! :) – Sanish Mar 13 '15 at 14:05

string replacement using awk command

2 Answers2

Linked