Extract specific keywords from XML file with bash script

Question

I have an XML file containing some entries characterized by specific kaywords. I need to run a for loop on the entries, extract two different keywords for each for them to be used in the for loop as variables.

Here is an example of list.xml:

<?xml version="1.0" encoding="UTF-8"?>
<responses type="C-FIND">
  <data-set xfer="1.2.840.10008.1.2.1" name="Little Endian Explicit">
    <element tag="0008,0005" vr="CS" vm="1" len="10" name="SpecificCharacterSet">ISO_IR 192</element>
    <element tag="0008,0052" vr="CS" vm="1" len="6" name="QueryRetrieveLevel">STUDY</element>
    <element tag="0008,0054" vr="AE" vm="1" len="8" name="RetrieveAETitle">PLATONE</element>
    <element tag="0010,0010" vr="PN" vm="1" len="16" name="PatientName">Anon^1600373003</element>
    <element tag="0020,000d" vr="UI" vm="1" len="42" name="StudyInstanceUID">1.3.76.13.99972.2.20181217085753.1484038.1</element>
  </data-set>
  <data-set xfer="1.2.840.10008.1.2.1" name="Little Endian Explicit">
    <element tag="0008,0005" vr="CS" vm="1" len="10" name="SpecificCharacterSet">ISO_IR 192</element>
    <element tag="0008,0052" vr="CS" vm="1" len="6" name="QueryRetrieveLevel">STUDY</element>
    <element tag="0008,0054" vr="AE" vm="1" len="8" name="RetrieveAETitle">PLATONE</element>
    <element tag="0010,0010" vr="PN" vm="1" len="16" name="PatientName">Anon^1599844862</element>
    <element tag="0020,000d" vr="UI" vm="1" len="42" name="StudyInstanceUID">1.3.76.13.99972.2.20180925142630.1456727.1</element>
  </data-set>
</responses>

I need to extract the keywords "PatientName" and "StudyInstanceUID". I tried to use something like this:

grep -A2 -i "PatientName" list.xml | while read -r string ; do
    PatientName="$(echo $string | grep -i "PatientName" | cut -d ">" -f 2 | cut -d "<" -f 1)"
    StudyInstanceUID="$(echo $string | grep -i "StudyInstanceUID" | cut -d ">" -f 2 | cut -d "<" -f 1)"
    echo "$PatientName"
    echo "$StudyInstanceUID"
done

The problem is that I obtain a lot of empty rows! What's the problem?

[EDIT] What I would like to obtain from this example is something like this:

Anon^1600373003
1.3.76.13.99972.2.20181217085753.1484038.1
Anon^1599844862
1.3.76.13.99972.2.20180925142630.1456727.1

Thanks so much.

Ivan

echo was just an example, I need to implement it in a bash script to perform other actions using the keywords. — I. Iudice, Nov 27 '20 at 13:26
@I.Iudice - Its unclear what you mean when you say _"I need to extract the keywords "PatientName" and "StudyInstanceUID""_. What are you actually trying to achieve? Is it to **A)** Assign e.g. `Anon^1600373003`, and `Anon^1599844862` to the `PatientName` variable, and similarly assign `1.3.76.13.99972.2.20181217085753.1484038.1` and `1.3.76.13.99972.2.20180925142630.1456727.1` to the `StudyInstanceUID` variable. **B)** Or, just print (echo) the results to the console like what you have shown? **3)** Or something else? — RobC, Nov 27 '20 at 15:12
I've been trying to achieve A), I have to use those variables to run signal processing algorithm. — I. Iudice, Nov 27 '20 at 15:42

markp-fuso · Accepted Answer · 2020-11-27T18:40:46.290

As Raman alluded to in the comment, using a XML-aware tool to parse XML data is probably your best bet especially if some of your XML may not be as formatted as displayed in the question (eg, everything on one long line).

Assumptions:

you can confirm all of your data will be formatted as like the samples in the question (ie, each element is on a separate line)
the search strings PatientName and StudyInstanceUID do not show up in larger strings (eg, LastPatientName or PreviousStudyInstanceUID)
the PatientName element is always listed before the StudyInstanceUID element

One awk solution which eliminates the need for all of the sub-process calls to echo, grep and cut:

awk -F'[<>]' '                                    # define input field separators as "<" and ">"
/PatientName/ || /StudyInstanceUID/ { print $3 }  # if we find one of our search strings then print field #3
' list.xml

The same as a one-liner, sans comments:

awk -F'[<>]' '/PatientName/ || /StudyInstanceUID/ { print $3 }' list.xml

The above generates:

Anon^1600373003
1.3.76.13.99972.2.20181217085753.1484038.1
Anon^1599844862
1.3.76.13.99972.2.20180925142630.1456727.1

As for capturing the output into variables (eg, within a while loop), we can make some small changes, eg:

awk -F'[<>]' '
/PatientName/      { pn=$3 }                      # store field #3 in variable "pn"
/StudyInstanceUID/ { printf "%s %s\n", pn, $3 }   # print data to stdout
' list.xml

This will generate:

Anon^1600373003 1.3.76.13.99972.2.20181217085753.1484038.1
Anon^1599844862 1.3.76.13.99972.2.20180925142630.1456727.1

Feeding this into a while loop:

while read -r PatientName StudyInstanceUID
do
    echo "+++++++++++++++++++"
    echo "PatientName:      ${PatientName}"
    echo "StudyInstanceUID: ${StudyInstanceUID}"
done < <(awk -F'[<>]' ' /PatientName/ { pn=$3 } /StudyInstanceUID/ { printf "%s %s\n", pn, $3 } ' list.xml)

And this generates:

+++++++++++++++++++
PatientName:      Anon^1600373003
StudyInstanceUID: 1.3.76.13.99972.2.20181217085753.1484038.1
+++++++++++++++++++
PatientName:      Anon^1599844862
StudyInstanceUID: 1.3.76.13.99972.2.20180925142630.1456727.1

I followed your advice; I substituted awk line with xmlstartlet pipe, and it works fine. Thanks a lot! — I. Iudice, Nov 28 '20 at 01:02

score 0 · Answer 2 · answered Nov 27 '20 at 14:24

Command:

grep -A2 -i "PatientName" list.xml

returns multiple lines:

    <element tag="0010,0010" vr="PN" vm="1" len="16" name="PatientName">Anon^1600373003</element>
    <element tag="0020,000d" vr="UI" vm="1" len="42" name="StudyInstanceUID">1.3.76.13.99972.2.20181217085753.1484038.1</element>
  </data-set>
--
    <element tag="0010,0010" vr="PN" vm="1" len="16" name="PatientName">Anon^1599844862</element>
    <element tag="0020,000d" vr="UI" vm="1" len="42" name="StudyInstanceUID">1.3.76.13.99972.2.20180925142630.1456727.1</element>
  </data-set>

so your while, read this output line by line. The result you get is correct because on line:

<element tag="0010,0010" vr="PN" vm="1" len="16" name="PatientName">Anon^1600373003</element>

StudyInstanceUID is not present and your variable will be empty.

In order to get the desired result, try this:

grep -A1 -i "PatientName" list.xml | while read -r string ; do
    PatientName="$(echo $string | grep -i "PatientName" | cut -d ">" -f 2 | cut -d "<" -f 1)"
    read string
    StudyInstanceUID="$(echo $string | grep -i "StudyInstanceUID" | cut -d ">" -f 2 | cut -d "<" -f 1)"
    echo "$PatientName"
    echo "$StudyInstanceUID"
    read string
done

Using read string you will get next line, but becareful, this works if lines are in that order.

score 0 · Answer 3 · answered Dec 05 '20 at 15:50

awk and sed are not designed to process XML. Please use a dedicated tool instead. I can recommend xidel.

Stdout:

$ xidel -s list.xml -e '
  //data-set/(
    element[@name="PatientName"],
    element[@name="StudyInstanceUID"]
  )
'
Anon^1600373003
1.3.76.13.99972.2.20181217085753.1484038.1
Anon^1599844862
1.3.76.13.99972.2.20180925142630.1456727.1

Variables:

$ xidel -s list.xml -e '
  //data-set/(
    eval(x"{concat("pn",position())}:=element[@name=""PatientName""]")[0],
    eval(x"{concat("si",position())}:=element[@name=""StudyInstanceUID""]")[0]
  )
'
pn1 := Anon^1600373003
si1 := 1.3.76.13.99972.2.20181217085753.1484038.1
pn2 := Anon^1599844862
si2 := 1.3.76.13.99972.2.20180925142630.1456727.1

These are internal variables that are just printed to stdout. Use --output-format=bash and Bash's built-in eval command to convert them to shell variables.

$ eval $(xidel -s list.xml -e '
  //data-set/(
    eval(x"{concat("pn",position())}:=element[@name=""PatientName""]")[0],
    eval(x"{concat("si",position())}:=element[@name=""StudyInstanceUID""]")[0]
  )
' --output-format=bash)

$ printf '%s\n' $pn1 $si1 $pn2 $si2
Anon^1600373003
1.3.76.13.99972.2.20181217085753.1484038.1
Anon^1599844862
1.3.76.13.99972.2.20180925142630.1456727.1

Extract specific keywords from XML file with bash script

3 Answers3