Parsing line and changing some text in place

Question

How do I parse a log file (not a full xml file, but it has some portion of xml data) for ExtData tags, which has some name-value pair, I need to mask it like this : For eg:

<ExtData>Name="Jason" Value="Special"</ExtData>
to
<ExtData>Name="Jason" Value="XXXXXXX"</ExtData>

I need to mask ExtData tag value like above only when Name is Jason or some set of name, and not for every Name.

For eg: if "DummyName" is not in set of names, than I do not want to change this below line.

<ExtData>Name="DummyName" Value="Garbage"</ExtData>

For eg: if "DummyName" is not in set of names, than I do not want to change this below line. (Please note that the value is "Jason")

<ExtData>Name="DummyName" Value="Jason"</ExtData>

For eg: if "DummyJasonName" is not in set of names, than I do not want to change this below line. (Note "Jason" in between "Dummy" and "Name")

<ExtData>Name="DummyJasonName" Value="Garbage"</ExtData>

I need to do all this in bash/shell script.

Bottom line is, I want to read a file, say, via sed/awk/match command. Check for ExtData tag in the line. If matched, Read the text between ExtData tag and /ExtData tag. In this multiline text, extract Name. If Name is from a set of names, then mask its corresponding "Value" data with equal number of 'X'.

Please let me know how to achieve the above task.

Update, the input line can actually span over multiple lines.

<ExtData>Name="Jason" 
Value="Special"
    </ExtData>

Or like this too:

<ExtData>
     Name="Jason" 
  Value="Special"
    </ExtData>

Thanks !! Puneet

Answer 1

In a bash shell, you can create a copy of the file with the info removed using this

sed 's#\(<ExtData>Name="Jason" Value="\).*\("</ExtData>\)#\1XXXXX\2#' xml.txt > xml_xxx.txt

Note that it's not the "official" way to change a xml file. Lots of format changes could occur that would render this script useless, but if you know that your XML file has 1 info per line formatted like that, it will work, exactly like for a text file and it's quick.

(also the question is tagged sed and bash, if it wasn't that would involve heavy xml parsing using libxml2, saxon or other libraries that can parse XML nodes)

Answer 2

To make the substitutions only for names Jason and Jim, try:

sed -E '/Jason|Jim/{:a; /Value=/bb; n; ba; :b; s/(Value="X*)[^X"]/\1X/; tb; }' file.xml

This command was tested on GNU sed. For BSD/OSX sed, some minor changes would be needed.

Example

Let's consider this test file:

$ cat file.xml
<ExtData>Name="Jason" Value="Special"</ExtData>
<ExtData>Name="DummyName" Value="Garbage"</ExtData>
<ExtData>Name="Jim"
    Value="OK"
        </ExtData>

Now, let's run our command:

$ sed -E '/Jason|Jim/{:a; /Value=/bb; n; ba; :b; s/(Value="X*)[^X"]/\1X/; tb; }' file.xml
<ExtData>Name="Jason" Value="XXXXXXX"</ExtData>
<ExtData>Name="DummyName" Value="Garbage"</ExtData>
<ExtData>Name="Jim"
    Value="XX"
        </ExtData>

How it works

• -E

  This tells set to use extended regular expressions.

• /Jason|Jim/{...}

  This tells sed to run the commands inside the curly braces only for lines that contain Jason or Jim. The command insides the braces breaks down into two parts:

  1. :a; /Value=/bb; n; ba;

     The first part reads lines until we find one that contains Value=. In more detail, :a defines a label a. /Value=/bb branches to label b if the current line contains Value=. If it doesn't, we print out the current line and read in the next one using the n command. We then branch (b) back to label a.

  2. :b; s/(Value="X*)[^X"]/\1X/; tb;

     This replaces the value with as many X as we need.

     In more detail, :b defines a label b. s/(Value="X*)[^X"]/\1X/ substitutes in the next X that we need after Value=. If a substitution was made (meaning that another X was needed), then the test command (t) tells sed to jump back to label b and we try again.

Restricting changes to within ExtData tags

Let's consider this more complex test file:

$ cat file2.xml
<Misc>Name="Jason" Value="DontChange"</Misc>
<ExtData>Name="Jason" Value="Special"</ExtData>
<Misc>Name="Jason" Value="DontChange"</Misc>
<ExtData>Name="DummyName" Value="DontChange"</ExtData>
<Misc>Name="Jason" Value="DontChange"</Misc>
<ExtData>Name="Jim"
    Value="OK"
        </ExtData>
<Misc>Name="Jason" Value="DontChange"</Misc>

To make the changes in ExtData tags but not the other tags, try:

$ sed -E '/[<]ExtData[>]/{:a; /Name=/{/Name="(Jason|Jim)"/!b}; /Value=/bb; n; ba; :b; s/(Value="X*)[^X"]/\1X/; tb; }' file2.xml
<Misc>Name="Jason" Value="DontChange"</Misc>
<ExtData>Name="Jason" Value="XXXXXXX"</ExtData>
<Misc>Name="Jason" Value="DontChange"</Misc>
<ExtData>Name="DummyName" Value="DontChange"</ExtData>
<Misc>Name="Jason" Value="DontChange"</Misc>
<ExtData>Name="Jim"
    Value="XX"
        </ExtData>
<Misc>Name="Jason" Value="DontChange"</Misc>

To do the above using a shell variable for the names:

names='Jason|Jim'
sed -E '/[<]ExtData[>]/{:a; /Name=/{/Name="'"$names"'"/!b}; /Value=/bb; n; ba; :b; s/(Value="X*)[^X"]/\1X/; tb; }' file2.xml

This substitutes the shell variable directly into the sed command. This should only be done this way if you trust the source of the shell variable.