0
file=mylog.log
search_str="&Name="
end_str="&"
sed -i -E ':a; s/('"$search_str"'X*)[^X'"$end_str"']/\1X/; ta' "$file"

Ex 1:
something&Name=JASON&else
to
something&Name=XXXXX&else

And actually, my current sed command works fine when instead of a '"$end_str"' if I use '&' character... Like this :

sed -i -E ':a; s/('"$search_str"'X*)[^X&]/\1X/; ta' "$file"

So, to summariz, it, after ^X if a single character comes than my given sed command works fine... But the same command does not work, if instead of character, i use a string...

For example, my sed command won't work in this case :
end_str="\%26"
sed -i -E ':a; s/('"$search_str"'X*)[^X'"$end_str"']/\1X/; ta' "$file"

Eg:

something&Name=JASON_MATTHEW_DONALD%26else
TO
something&Name=XXXXXXXXXXXXXXXXXXXX%26else

Eg:2

something&Name=JASON%26else
TO
something&Name=XXXXX%26else

Please let me know

Puneet Jain
  • 97
  • 1
  • 10
  • 1
    In `sed`, `[ ]` is a character class, not a string. It means any single character within `[ ]`. With the `^` prefix, it means any single character that is not in `[ ]`. – alvits Jan 26 '17 at 03:07
  • @alvits Thanks Alvits... So, instead of [] what i should use? – Puneet Jain Jan 26 '17 at 04:44
  • @PuneetJain: Can you summarise your final input and expected output? It is not clear exactly as of now. – Inian Jan 26 '17 at 05:23
  • `` `something&Name=JASON%26else` `TO` `something&Name=XXXXX%26else` `` – Puneet Jain Jan 26 '17 at 08:12

3 Answers3

0

Place your string variable outside the character class and capture it to check for further substitutions:

sed -i -E ':a; s/('"$search_str"'X*)[^X](.*'"$end_str"')/\1X\2/; ta' "$file"
SLePort
  • 15,211
  • 3
  • 34
  • 44
  • Unfortunately it didn;t worked.It replaced the & with X and eaten many characters in between... May be fine tune a little bit the above sed command? – Puneet Jain Jan 26 '17 at 08:25
  • Try to escape the `&`: `\&`. Does it work with `\%26` ? – SLePort Jan 26 '17 at 09:16
  • I tried with \%26. Didn't worked... Single character & works with my orif=ginal sed command, so I am not worried about it... but with \%26, mine and your sed command didn't worked...any pointers? – Puneet Jain Jan 26 '17 at 09:27
  • I tested with your sample inputs (single or multiple characters), it's working with GNU sed. What's your OS? your sed version? – SLePort Jan 26 '17 at 12:35
  • `can u try this if` `search_str="\%3dName="` `end_str="\%26"` `something%3dName=JASON%MATTHEW%2DONALD%26else` `TO` `something%3dName=XXXXXXXXXXXXXXXXXXXXXX%26else` – Puneet Jain Jan 26 '17 at 15:38
  • That's my input with this command. – SLePort Jan 26 '17 at 15:43
  • what is the output ? – Puneet Jain Jan 26 '17 at 15:46
  • `something%3dName=XXXXXXXXXXXXXXXXXXXXXX%26else`| – SLePort Jan 26 '17 at 15:52
  • OS: Windows ..... sed (GNU sed) 4.2.2 I am not sure why you sed command not perfactly working in my setup... Do you think we should tweak the command a little bit for my OS/GNU/SED – Puneet Jain Jan 26 '17 at 16:36
  • If you're under Windows, it might be carriage return in your input file that you can remove with `tr -d '\r' < input.log > newinput.log` – SLePort Jan 27 '17 at 10:00
0

If the number of X doesn't matter you can simplify it to:

search="Name"
sed "s/$search=[^&]*/$search=XXX/" input.file

This assumes that $search won't contain special characters which have a meaning in sed's regex syntax. If special characters can be a problem you need to prepare the $search variable, as explained here: Is it possible to escape regex metacharacters reliably with sed

Community
  • 1
  • 1
hek2mgl
  • 152,036
  • 28
  • 249
  • 266
  • Unfortunately number of X does matter... ::-( – Puneet Jain Jan 26 '17 at 08:26
  • Can you explain why? – hek2mgl Jan 26 '17 at 08:29
  • Because I need to replace the characters with two strings, with equal number of X. See my edited question example 1 and 2 – Puneet Jain Jan 26 '17 at 08:35
  • I'm just asking why. I mean why do you need to preserve the length of the user name? – hek2mgl Jan 26 '17 at 08:50
  • Because other than names, i need to replace passwords and some other fields.. for which i need to know how many characters i am replacing – Puneet Jain Jan 26 '17 at 09:01
  • 1
    That's exactly my point. When you replace passwords you *don't* want to preserve the length. Because the length is already a *very* valuable information to brute force a password. Replace all passwords by tree *. That's fine. – hek2mgl Jan 26 '17 at 09:32
  • PS: Sending passwords in plain text via an url is another point, but that's off-topic here. – hek2mgl Jan 26 '17 at 09:33
  • No.. because... for other fields we receive from clients, some sensitive info, we need to mask it by X (euall number of X) so that we can debug later as to how many bytes were sent for a field – Puneet Jain Jan 26 '17 at 09:34
0

I insist on the point that keeping the length of passwords in logs is a very bad idea (security wise).

Having said that:

First, a character list [...] is not the right tool to match an string. For that we need to use an alternating value (...|...).

end_str="(&|%26)"

But it is quite difficult to express a "not a string" in regex. 

not_end_str="([^&]|[^%]|%[^2]|%2[^6])"

Using all that we may build a pure bash solution (maybe not fast, but works).
It prints to stdout to show how it works. Redirect to a file to store the result.

file=mylog.log
search_str="&Name="
end_str="(&|%26)"                       # write end_str as an alternate value.
not_end_str="([^&]|[^%]|%[^2]|%2[^6])"  # regex negate end_string.

# Build a regex that split each part of the text.
myreg="(.*${search_str}X*)(${not_end_str}*)([&%].*)"

while IFS=$'\n' read line; do
    [[ $line =~ $myreg ]]
    len=$((${#BASH_REMATCH[@]}-2))          # do not count [0] and last.
    arr=("${BASH_REMATCH[@]:1}")            # remove [0] and last.
    arr[1]=${arr[1]//?/X}                   # Replace name with "X"'s.
    arr[2]=''                               # Clear not_end_str match
    printf '%s' "${arr[@]}"; echo           # Print modified line.
done <"$file"

Further reading: