Saving specific parts from REST API into list

Question

I'm currently working on a script, that's getting a full list of network devices from our Cisco Prime Infrastructure and then adding these devices to our Cisco ISE installation.

The output data from our Cisco Prime Infrastructure is over 25000 lines as text.

Example output:

  <entity dtoType="devicesDTO" type="Devices" url="https://X.X.X.X/webacs/api/v4/data/Devices/7860155308">
    <devicesDTO displayName="7860155308" id="7860155308">
      <collectionDetail>&lt;status&gt;&lt;general code="SUCCESS"/&gt;&lt;/status&gt;</collectionDetail>
      <collectionTime>2019-06-28T01:02:32.494+02:00</collectionTime>
      <creationTime>2018-10-05T08:45:01.461+02:00</creationTime>
      <deviceId>7849390591</deviceId>
      <deviceName>008-HX.TEST.SWITCH-CORE-01</deviceName>
      <deviceType>Cisco Catalyst38xx stack-able ethernet switch</deviceType>
      <ipAddress>172.29.0.254</ipAddress>
      <location>Test Location</location>
      <managementStatus>MANAGED_AND_SYNCHRONIZED</managementStatus>
      <productFamily>Switches and Hubs</productFamily>
      <reachability>REACHABLE</reachability>
      <softwareType>IOS-XE</softwareType>
      <softwareVersion>16.6.5</softwareVersion>
      <adminStatus>MANAGED</adminStatus>
      <collectionStatus>COMPLETED</collectionStatus>
      <manufacturerPartNrs>
        <manufacturerPartNr>
          <partNumber>WS-C3850-12XS-E</partNumber>
        </manufacturerPartNr>
      </manufacturerPartNrs>
    </devicesDTO>
  </entity>
  <entity dtoType="devicesDTO" type="Devices" url="https://X.X.X.X/webacs/api/v4/data/Devices/7605631035">
    <devicesDTO displayName="7605631035" id="7605631035">
      <collectionDetail>&lt;status&gt;&lt;general code="SUCCESS"/&gt;&lt;/status&gt;</collectionDetail>
      <collectionTime>2019-06-28T00:45:33.177+02:00</collectionTime>
      <creationTime>2018-09-04T01:48:15.790+02:00</creationTime>
      <deviceId>7598542980</deviceId>
      <deviceName>008-HX.TEST.SWITCH-UA-01.Vejle.dk</deviceName>
      <deviceType>Cisco Catalyst 29xx Stack-able Ethernet Switch</deviceType>
      <ipAddress>172.29.0.2</ipAddress>
      <location>Test Location</location>
      <managementStatus>MANAGED_AND_SYNCHRONIZED</managementStatus>
      <productFamily>Switches and Hubs</productFamily>
      <reachability>REACHABLE</reachability>
      <softwareType>IOS</softwareType>
      <softwareVersion>15.2(2)E9</softwareVersion>
      <adminStatus>MANAGED</adminStatus>
      <collectionStatus>COMPLETED</collectionStatus>
      <manufacturerPartNrs>
        <manufacturerPartNr>
          <partNumber>WS-C2960X-48FPD-L</partNumber>
        </manufacturerPartNr>
      </manufacturerPartNrs>
    </devicesDTO>
  </entity>
  <entity dtoType="devicesDTO" type="Devices" url="https://X.X.X.X/webacs/api/v4/data/Devices/7605631034">
    <devicesDTO displayName="7605631034" id="7605631034">
      <collectionDetail>&lt;status&gt;&lt;general code="SUCCESS"/&gt;&lt;/status&gt;</collectionDetail>
      <collectionTime>2019-06-28T00:46:35.393+02:00</collectionTime>
      <creationTime>2018-09-04T01:48:00.776+02:00</creationTime>
      <deviceId>7598542967</deviceId>
      <deviceName>008-HX.TEST.SWITCH-UA-02.Vejle.dk</deviceName>
      <deviceType>Cisco Catalyst 29xx Stack-able Ethernet Switch</deviceType>
      <ipAddress>172.29.0.3</ipAddress>
      <location>Test Location</location>
      <managementStatus>MANAGED_AND_SYNCHRONIZED</managementStatus>
      <productFamily>Switches and Hubs</productFamily>
      <reachability>REACHABLE</reachability>
      <softwareType>IOS</softwareType>
      <softwareVersion>15.2(2)E9</softwareVersion>
      <adminStatus>MANAGED</adminStatus>
      <collectionStatus>COMPLETED</collectionStatus>
      <manufacturerPartNrs>
        <manufacturerPartNr>
          <partNumber>WS-C2960X-48FPD-L</partNumber>
        </manufacturerPartNr>
      </manufacturerPartNrs>
    </devicesDTO>
  </entity>
  <entity dtoType="devicesDTO" type="Devices" url="https://X.X.X.X/webacs/api/v4/data/Devices/7605631033">
    <devicesDTO displayName="7605631033" id="7605631033">
      <collectionDetail>&lt;status&gt;&lt;general code="SUCCESS"/&gt;&lt;/status&gt;</collectionDetail>
      <collectionTime>2019-06-28T00:47:01.412+02:00</collectionTime>
      <creationTime>2018-09-04T01:47:45.817+02:00</creationTime>
      <deviceId>7598542954</deviceId>
      <deviceName>008-UX1.TEST.SWITCH-UA-01.Vejle.dk</deviceName>
      <deviceType>Cisco Catalyst 29xx Stack-able Ethernet Switch</deviceType>
      <ipAddress>172.29.0.4</ipAddress>
      <location>Test Location</location>
      <managementStatus>MANAGED_AND_SYNCHRONIZED</managementStatus>
      <productFamily>Switches and Hubs</productFamily>
      <reachability>REACHABLE</reachability>
      <softwareType>IOS</softwareType>
      <softwareVersion>15.2(2)E9</softwareVersion>
      <adminStatus>MANAGED</adminStatus>
      <collectionStatus>COMPLETED</collectionStatus>
      <manufacturerPartNrs>
        <manufacturerPartNr>
          <partNumber>WS-C2960X-48FPD-L</partNumber>
        </manufacturerPartNr>
      </manufacturerPartNrs>
    </devicesDTO>
  </entity>
  <entity dtoType="devicesDTO" type="Devices" url="https://X.X.X.X/webacs/api/v4/data/Devices/7605631038">
    <devicesDTO displayName="7605631038" id="7605631038">
      <collectionDetail>&lt;status&gt;&lt;general code="SUCCESS"/&gt;&lt;/status&gt;</collectionDetail>
      <collectionTime>2019-06-28T00:47:25.438+02:00</collectionTime>
      <creationTime>2018-09-04T01:49:01.038+02:00</creationTime>
      <deviceId>7598543019</deviceId>
      <deviceName>008-UX2.TEST.SWITCH-UA-01.Vejle.dk</deviceName>
      <deviceType>Cisco Catalyst 35xx Stack-able Ethernet Switch</deviceType>
      <ipAddress>172.29.0.5</ipAddress>
      <location>Test Location</location>
      <managementStatus>MANAGED_AND_SYNCHRONIZED</managementStatus>
      <productFamily>Switches and Hubs</productFamily>
      <reachability>REACHABLE</reachability>
      <softwareType>IOS</softwareType>
      <softwareVersion>15.2(4)E7</softwareVersion>
      <adminStatus>MANAGED</adminStatus>
      <collectionStatus>COMPLETED</collectionStatus>
      <manufacturerPartNrs>
        <manufacturerPartNr>
          <partNumber>WS-C3560CX-12PD-S</partNumber>
        </manufacturerPartNr>
      </manufacturerPartNrs>
    </devicesDTO>
  </entity>
  <entity dtoType="devicesDTO" type="Devices" url="https://X.X.X.X/webacs/api/v4/data/Devices/7605631037">
    <devicesDTO displayName="7605631037" id="7605631037">
      <collectionDetail>&lt;status&gt;&lt;general code="SUCCESS"/&gt;&lt;/status&gt;</collectionDetail>
      <collectionTime>2019-06-28T00:45:32.775+02:00</collectionTime>
      <creationTime>2018-09-04T01:48:45.760+02:00</creationTime>
      <deviceId>7598543006</deviceId>
      <deviceName>008-UX3.TEST.SWITCH-UA-01.Vejle.dk</deviceName>
      <deviceType>Cisco Catalyst 29xx Stack-able Ethernet Switch</deviceType>
      <ipAddress>172.29.0.6</ipAddress>
      <location>Test Location</location>
      <managementStatus>MANAGED_AND_SYNCHRONIZED</managementStatus>
      <productFamily>Switches and Hubs</productFamily>
      <reachability>REACHABLE</reachability>
      <softwareType>IOS</softwareType>
      <softwareVersion>15.2(2)E9</softwareVersion>
      <adminStatus>MANAGED</adminStatus>
      <collectionStatus>COMPLETED</collectionStatus>
      <manufacturerPartNrs>
        <manufacturerPartNr>
          <partNumber>WS-C2960X-24PD-L</partNumber>
        </manufacturerPartNr>
      </manufacturerPartNrs>
    </devicesDTO>
  </entity>

Now i only want to save the lines starting with entity and ending with /entity into the list and each section should be a list of its own.

Example:

[['<entity dtoType="devicesDTO" type="Devices" url="https://X.X.X.X/webacs/api/v4/data/Devices/7860155308">', '...truncated...', '</entity>'], ['<entity dtoType="devicesDTO" type="Devices" url="https://X.X.X.X/webacs/api/v4/data/Devices/7605631035">', '...truncated...', '</entity>']]

My current code:

#!/usr/bin/python
# coding=utf-8
import sys
import requests
import xml.etree.ElementTree as ET

response = requests.get('https://X.X.X.X/webacs/api/v4/data/Devices?.full=true&.sort=deviceName', verify=False)
output = response.text

data = []

for line in output:
    data.append(line)

But it's giving weird results. I cannot fathom how to solve this currently. Some help would be appreciated.

Edit:

I can see there's a \n at the end of every line. If i do this:

output = output.split("\n")

I get a single list containing all the data.

It looks like an xml format, so look into reading xml with python — Peter, Jun 28 '19 at 11:04
@Peter Thanks i will look into that. I thought treating it as a normal string would be easier to edit and work with. Judging by the lack of replies must mean, that i am wrong. — Cow, Jun 28 '19 at 11:11
If it's something like that, then you might as well use the tools at your disposal. It would be easy to parse the text if you're just looking for `` and ``, but there's also parameters inside it which would be a bit more messy to deal with. — Peter, Jun 28 '19 at 11:17

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

You can take two approaches:

1st Approach: Line by line

If you want to have your code run through it line-by-line, you can check if the line matches your requirements, using a regex:

for line in output:
    if re.match(r'entity.*/entity', line):
        print("It's a match")

This will match every line where you have "any text (like <) entity any text /entity any text (like >)" However, this will only capture single line entities. Alternatively, you can find lines with a starting tag and opening tag, and capture content in between:

for line in output:
    if re.match(r'entity', line):
        if re.match(r'/entity', line):
            print("End of block")
        else:
            print("Start of block")

2nd Approach: XML

If you want to actually treat it like an XML, not only look for single line entity objects, you can use xml.etree.ElementTree and run a recursive findall as described here to find every entity element.

Saving specific parts from REST API into list

1 Answers1