convert json to a data frame like output in python

Question

I have a script to pull data from CA Introscope. The script prints out the values as this:

print data

[(TimesliceGroupedMetricData){
   metricData[] = 
      (MetricData){
         agentName = "web01|WEB|instance_01"
         metricName = "WebSpherePMI|orbPerfModule:ConcurrentRequestCount"
         metricType = 258
         metricValue = "0"
      },
      (MetricData){
         agentName = "app01|APP|instnace_03"
         metricName = "WebSpherePMI|beanModule|mobile.jar:ReturnsToPoolCount"
         metricType = 258
         metricValue = "3"
      },
      (MetricData){
         agentName = "app02|APP|instance_02"
         metricName = "JSP|add_client:Stall Count"
         metricType = 385
         metricValue = "0"
      },
      (MetricData){
         agentName = "web05|WEB|instance_02"
         metricName = "WebSpherePMI|beanModule|bizlogic#bizlogic.jar|ejb.entity|WorkStepInstanceEBBBean:ReadyCount"
         metricType = 258
         metricValue = "0"
      },
   timesliceEndTime = 2015-01-05 16:33:15
   timesliceStartTime = 2015-01-05 16:28:15


}]

I need this output to print out as this:

metricName, AppName, ServerName, InstanceName,Value, timesliceEndTime

actual data:

WebSpherePMI|orbPerfModule:ConcurrentRequestCount, WEB, web01,instance_01, 0, 2015-01-05 16:33:15
WebSpherePMI|beanModule|mobile.jar:ReturnsToPoolCount,APP,app01,instance_02,3, 2015-01-05 16:33:15
JSP|add_client:Stall Count, APP, app02,instance_02,0, 2015-01-05 16:33:15
WebSpherePMI|beanModule|bizlogic#bizlogic.jar|ejb.entity|WorkStepInstanceEBBBean:ReadyCount, WEB, web05, insntance02, 0, 2015-01-05 16:33:15

I place comma to separate the field output, but is not necessary. Can somebody give me some guide line how to do this in python?

@MightyPork, you're right sorry. This is the structure of data that I get. — user1471980, Jan 05 '15 at 21:53
@MattDMo, I just updated the post. I am getting the data from CA Introscope. — user1471980, Jan 05 '15 at 22:01

score 2 · Accepted Answer · answered Jan 06 '15 at 16:42

Okay so here it is. I've made a dummy class and filled it with your data, so it is tested against that, YMMV.

def dumpDataBlob(blob):
    """ Print data object in required format """

    timesliceEndTime = blob.timesliceEndTime

    for entry in blob.metricData:
        # gather values
        metricName = entry.metricName
        metricValue = entry.metricValue
        (serverName, appName, instanceName) = entry.agentName.split('|')

        print('{met},{app},{ser},{ins},{val},{tim}'.format(
            met=metricName,
            app=appName,
            ser=serverName,
            ins=instanceName,
            val=metricValue,
            tim=timesliceEndTime))

Here's my test classes, fyi

class MetricData:
    def __init__(self, agentName, metricName, metricType, metricValue):
        self.agentName = agentName
        self.metricName = metricName
        self.metricType = metricType
        self.metricValue = metricValue

class DummyObject:
    metricData = []

    def __init__(self):
        self.metricData = []

        self.metricData.append(MetricData(
            agentName = "web01|WEB|instance_01",
            metricName = "WebSpherePMI|orbPerfModule:ConcurrentRequestCount",
            metricType = 258,
            metricValue = "0"
        ))

        self.metricData.append(MetricData(
            agentName = "app01|APP|instnace_03",
            metricName = "WebSpherePMI|beanModule|mobile.jar:ReturnsToPoolCount",
            metricType = 258,
            metricValue = "3"
        ))

        self.metricData.append(MetricData(
            agentName = "app02|APP|instance_02",
            metricName = "JSP|add_client:Stall Count",
            metricType = 385,
            metricValue = "0"
        ))

        self.metricData.append(MetricData(
            agentName = "web05|WEB|instance_02",
            metricName = "WebSpherePMI|beanModule|bizlogic#bizlogic.jar|ejb.entity|WorkStepInstanceEBBBean:ReadyCount",
            metricType = 258,
            metricValue = "0"
        ))

        self.timesliceEndTime = "2015-01-05 16:33:15"
        self.timesliceStartTime = "2015-01-05 16:28:15"

@MightPork, I am sorry but not following this. The whole data can be assigned to a variable called data and I need to parse data to get the desired output. — user1471980, Jan 06 '15 at 17:23
that's what you pass into the function I provided. The classes are just what I used to test if it works. I don't have the actual data you want to parse, so I couldn't check if it works with those. This is my best guess. — MightyPork, Jan 06 '15 at 17:24
If I do this dumpDataBlob(data), would what work, given the data as the formated in the orginal post. I am new to this, please bear with me. — user1471980, Jan 06 '15 at 17:33
I tried your function dumpDataBlob(data), I get this error:AttributeError: 'list' object has no attribute 'timesliceEndTime' — user1471980, Jan 06 '15 at 17:42
You showed only the opening bracket of the list, so I'm not sure what is really in it. Regardless, try `dumpDataBlob(data[0])` - if there is only one item, it will do it. — MightyPork, Jan 06 '15 at 17:47
@MightyPort, yes, sorry, I missed the ending bracket. I just updated the post. — user1471980, Jan 06 '15 at 17:52

convert json to a data frame like output in python

1 Answers1