1

I want to change (via bash script using SED command) all the characters in between two given strings with an equal number of X characters. If say, the first string is %26Name%3d and the last sting is %26, then anything in between these two strings I want to replace with an equal number of X characters. I am reading a file in place. The file is roughly 1MB.

So 

SOMETHING%26Name%3dTHISTEXTNEEDSTOBE%26ELSE

to 

SOMETHING%26Name%3dXXXXXXXXXXXXXXXXX%26ELSE

OR if start string is %26Last%3d and ending string is %26 

something%26Last%3d%2%2%4%2%4%3%5%%2%2%2%26else

to

something%26Last%3dXXXXXXXXXXXXXXXXXXXXX%26else

I tried below using the sed command in my bash script but didn't worked perfectly. 

file=mylog.log
myAnd=\\%26
myEqual=\\%3d
startList=Name\|Last
end_str="\%26"
search_str="$myAnd""(""$startList"")""$myEqual"

sed -i -E ':a; s/('"$search_str"'X*)[^X](.*'"$end_str"')/\1X\2/; ta' "$file"

PS startList=Name\|Last above declaration and also of myAnd and myEqual.

Example:

SOME%26Name%3dTHISTEXTNEEDSTOBE%26Last%3dDUTTA  
to  
SOME%26Name%3dXXXXXXXXXXXXXXXXX%26Last%3dXXXXX
Puneet Jain
  • 97
  • 1
  • 10

3 Answers3

2

Here is a sed solution:

sed -e :a -e 's/\(%26Name%3d[X]*\)[^X]\(.*%26\)/\1X\2/;ta'
Beta
  • 96,650
  • 16
  • 149
  • 150
  • Thanks. I will try your suggested solution... But if you don't mind, could you please give me solution using **startList** and **end_str** variables please? The reason is that i have approx 15-20 items in the startList, and i do not want to repeat the sed command 15-20 for each item. Thanks in advance. – Puneet Jain Jan 27 '17 at 01:29
  • thanks.. But its failing if the data is like this 1. `SOMETHING%26Name%3dTHISTEXTNEEDSTOBE` basically **end of the line** is NOT treated as a stop point. And 2. `something%26Last%3d%2%2%4%2%4%3%5%%2%2%2%26Name%3d%%%%%26` basically if more than 1 occurance of "start" elements come, then it does not do it correctly – Puneet Jain Jan 28 '17 at 04:58
  • @PuneetJain: You have now substantially changed the question *three times* since I answered it. (And it is impossible to guess what you consider "correct" behavior when more than one start element appears in the string.) That is not the correct way to use this site. Try to understand this solution, and if you find that there is something else you can't figure out how to do, reduce it to a minimal example and ask it as a new question. – Beta Jan 29 '17 at 01:38
  • Sorry @Beta. I will guarantee that I will ask the question from the next time to the concentrated area. Sorry again... – Puneet Jain Jan 29 '17 at 01:45
  • If i may ask you, do you mind giving the solution to this updated question.. This is the final edit of the question, i promise. – Puneet Jain Jan 29 '17 at 01:45
  • @PuneetJain: **Try to understand the solution I've already given.** Adapting it to the updated question is easy, once you understand it. – Beta Jan 29 '17 at 02:10
  • OK I will try to adapt it based on my updated question, but would you mind explaining me your answer in little bit detail please so that i can understand it and accordingly will try to adapt to my needs? Your current answer is not replacing chars with 'X' till the immediate next stop point, but it is replacing till the last stop point. Not sure currently how I can adapt it. Please help. Would appreciate it!! – Puneet Jain Jan 29 '17 at 18:22
  • As per your suggestion, I raised a new question http://stackoverflow.com/questions/41944021/sed-substitude-all-characters-between-two-strings-by-char-x If you could also look at it, I would really appreciate.. Thanks in advance. – Puneet Jain Jan 30 '17 at 20:34
  • thanks again for all your inputs.. this http://stackoverflow.com/questions/41944021/sed-substitude-all-characters-between-two-strings-by-char-x finally worked for me !! – Puneet Jain Jan 31 '17 at 06:25
1

Hope this is what you are looking for.

As you have mentioned only the startList is different, I have framed the code to only vary this. Add the rest of the variables to the array in single quotes just like the two I have added.

Script:

file=/path/to/your/file
declare -a arr=('%26Name%3d' '%26Last%3d')
for i in "${arr[@]}";
do
sed -i -e :a -e "s/\($i[X]*\)[^X]\(.*%26\)/\1X\2/;ta" $file;
done

Detailed Session Output:

$ cat script.sh
file=/home/siju/Desktop/test
declare -a arr=('%26Name%3d' '%26Last%3d')
for i in "${arr[@]}";
do
sed -i -e :a -e "s/\($i[X]*\)[^X]\(.*%26\)/\1X\2/;ta" $file;
done
$ 
$ cat test
SOMETHING%26Name%3dTHISTEXTNEEDSTOBE%26ELSE  
something%26Last%3d%2%2%4%2%4%3%5%%2%2%2%26else 
$ 
$ ./script.sh 
$ cat test
SOMETHING%26Name%3dXXXXXXXXXXXXXXXXX%26ELSE  
something%26Last%3dXXXXXXXXXXXXXXXXXXXXX%26else
Siju V
  • 143
  • 1
  • 5
  • thanks.. But its failing if the data is like this 1. `SOMETHING%26Name%3dTHISTEXTNEEDSTOBE` basically **end of the line** is NOT treated as a stop point. And 2. `something%26Last%3d%2%2%4%2%4%3%5%%2%2%2%26Name%3d%%%%%26` basically if **more than 1 occurance of "start" elements** come, then it does not do it correctly – Puneet Jain Jan 28 '17 at 05:06
0

perl's s/// operator lets you evaluate perl code in the replacement part with the e flag

perl -e '
    my @strings = qw{
        SOMETHING%26Name%3dTHISTEXTNEEDSTOBE%26ELSE
        something%26Last%3d%2%2%4%2%4%3%5%%2%2%2%26else
    };
    my $start = q{%26(?:Name|Last)%3d};
    my $stop = q{%26};
    for my $str (@strings) {
        (my $copy = $str) =~ s/$start\K(.*?)(?=$stop)/"X" x length($1)/ge;
        print "$str\n$copy\n";
    }
'

SOMETHING%26Name%3dTHISTEXTNEEDSTOBE%26ELSE
SOMETHING%26Name%3dXXXXXXXXXXXXXXXXX%26ELSE
something%26Last%3d%2%2%4%2%4%3%5%%2%2%2%26else
something%26Last%3dXXXXXXXXXXXXXXXXXXXXX%26else
glenn jackman
  • 238,783
  • 38
  • 220
  • 352
  • Thanks glenn, but unfortunately, I am reading the file via sed command only in my bash script and i need to replace the text in file only. Could you give me solution for bash environment please? – Puneet Jain Jan 27 '17 at 00:56
  • 1
    I don't know how to do it in sed. Check the manual: https://www.gnu.org/software/sed/manual/sed.html -- and a perl solution is a valid "solution for bash environment" – glenn jackman Jan 27 '17 at 01:02