Possible Parser for Unknown String Format(soup?) from SUDS.client

Question

I am using suds package to query a API from a website, the data returned from their website looks like this,:

(1). Can anyone tell me what kind of format is this?

(2). If so, what will be the easiest way to parse the data looks like this? I have dealt quite a lot with HTML/XML format using BeautifulSoup but before I lift my finger to write regular expressions for this type of format. I am curious is this some type of 'popular format' and there are actually some beautiful parser already written. Thanks.

    # Below are the header and tail of the response.. 
    (DetailResult)
{ 

status = (Status){ message = None code = "0" } 

searchArgument = (DetailSearchArgument){ reqPartNumber = "BQ" reqMfg = "T" reqCpn = None } 

detailsDto[] = (DetailsDto){ 

summaryDto = (SummaryDto){ PartNumber = "BQ" seMfg = "T" description = "Fast" } 

packageDto[] = 
(PackageDto){ fetName = "a" fetValue = "b" }, 
(PackageDto){ fetName = "c" fetValue = "d" }, 
(PackageDto){ fetName = "d" fetValue = "z" }, 
(PackageDto){ fetName = "f" fetValue = "Sq" }, 
(PackageDto){ fetName = "g" fetValue = "p" }, 

additionalDetailsDto = (AdditionalDetailsDto){ cr = None pOptions = None inv = None pcns = None } 

partImageDto = None 

riskDto = (RiskDto){ life= "Low" lStage = "Mature" yteol = "10" Date = "2023"} 

partOptionsDto[] = (ReplacementDto){ partNumber = "BQ2" manufacturer = "T" type = "Reel" }, 

inventoryDto[] = 
(InventoryDto){ distributor = "V" quantity = "88" buyNowLink = "https://www..." }, 
(InventoryDto){ distributor = "R" quantity = "7" buyNowLink = "http://www.r." }, 
(InventoryDto){ distributor = "RS" quantity = "2" buyNowLink = "http://www.rs.." }, 
}, 

}

Are these all the data that are recieved? Because it looks like it is badly formatted, there are unmatching brackets and quotes. — rlms, Nov 21 '13 at 23:10
@sweeneyrod Sorry, these are only the head and tail of this long string, I thought these info should be enough for the people who knows what type of format will recognize it. — B.Mr.W., Nov 21 '13 at 23:13

PaulMcG · Accepted Answer · 2013-11-23T08:16:44.377

This looks like some kind of nested repr output, similar to JSON but with structure or object name information ("a Status contains a message and a code"). If it's nested, regexes alone won't do the job. Here is a rough pass at a pyparsing parser

sample = """
... given sample text ...
 """
from pyparsing import *

# punctuation
LPAR,RPAR,LBRACE,RBRACE,LBRACK,RBRACK,COMMA,EQ = map(Suppress,"(){}[],=")

identifier = Word(alphas,alphanums+"_")

# define some types that can get converted to Python types
# (parse actions will do conversion at parse time)
NONE = Keyword("None").setParseAction(replaceWith(None))
integer = Word(nums).setParseAction(lambda t:int(t[0]))
quotedString.setParseAction(removeQuotes)

# define a placeholder for a nested object definition (since objDefn
# will be referenced within its own definition)
objDefn = Forward()

objType = Combine(LPAR + identifier + RPAR)

objval = quotedString | NONE | integer | Group(objDefn)
objattr = Group(identifier + EQ + objval)

arrayattr = Group(identifier + LBRACK + RBRACK + EQ + Group(OneOrMore(Group(objDefn)+COMMA)) )

# use '<<' operator to assign content to previously declared Forward
objDefn << objType + LBRACE + ZeroOrMore((arrayattr | objattr) + Optional(COMMA)) + RBRACE
# parse sample text
result = objDefn.parseString(sample)

# use pprint to list out indented parsed data
import pprint
pprint.pprint(result.asList())

Prints:

['DetailResult',
 ['status', ['Status', ['message', None], ['code', '0']]],
 ['searchArgument',
  ['DetailSearchArgument',
   ['reqPartNumber', 'BQ'],
   ['reqMfg', 'T'],
   ['reqCpn', None]]],
 ['detailsDto',
  [['DetailsDto',
    ['summaryDto',
     ['SummaryDto',
      ['PartNumber', 'BQ'],
      ['seMfg', 'T'],
      ['description', 'Fast']]],
    ['packageDto',
     [['PackageDto', ['fetName', 'a'], ['fetValue', 'b']],
      ['PackageDto', ['fetName', 'c'], ['fetValue', 'd']],
      ['PackageDto', ['fetName', 'd'], ['fetValue', 'z']],
      ['PackageDto', ['fetName', 'f'], ['fetValue', 'Sq']],
      ['PackageDto', ['fetName', 'g'], ['fetValue', 'p']]]],
    ['additionalDetailsDto',
     ['AdditionalDetailsDto',
      ['cr', None],
      ['pOptions', None],
      ['inv', None],
      ['pcns', None]]],
    ['partImageDto', None],
    ['riskDto',
     ['RiskDto',
      ['life', 'Low'],
      ['lStage', 'Mature'],
      ['yteol', '10'],
      ['Date', '2023']]],
    ['partOptionsDto',
     [['ReplacementDto',
       ['partNumber', 'BQ2'],
       ['manufacturer', 'T'],
       ['type', 'Reel']]]],
    ['inventoryDto',
     [['InventoryDto',
       ['distributor', 'V'],
       ['quantity', '88'],
       ['buyNowLink', 'https://www...']],
      ['InventoryDto',
       ['distributor', 'R'],
       ['quantity', '7'],
       ['buyNowLink', 'http://www.r.']],
      ['InventoryDto',
       ['distributor', 'RS'],
       ['quantity', '2'],
       ['buyNowLink', 'http://www.rs..']]]]]]]]

Hi, your answer is so inspiring.. I just posted the complete API response there and in case it will be trivial step to get the answer from you. Actually, I noticed there are two types there `A=(A){A=""}` and `A=(){},(){},..,` I am wondering that `pyparsing` will be enough to handle that. — B.Mr.W., Nov 22 '13 at 23:51
With your help, now I can see that the weird API format has already been perfectly translated into python list object.. However, I am thinking that dictionary in python would be a better fit unless it is col[]=(A){},(A){}.. in which case has to be array.. I don't know if pyparsing could easily handle it... Right now, it is still difficult to query the result. I am also doing some research meanwhile. — B.Mr.W., Nov 23 '13 at 15:37
I figured this was coming. Yes, pyparsing allows you to attach names to results, or to dynamically name results using values found in the parsed text using the Dict class. — PaulMcG, Nov 23 '13 at 15:43
thanks for the effort already put into this question, that is really inspiring and I have learned a lot from your answer. — B.Mr.W., Nov 23 '13 at 16:06

Possible Parser for Unknown String Format(soup?) from SUDS.client

1 Answers1

Linked