How to convert an xml to a dictionary with some modifications?

Question

I currently have an xml file in the following format:

<?xml version="1.0" encoding="UTF-8" ?>
    <Garden>
        <id>97</id>
        <Flowers>
            <id>98</id>
            <Type>
                <id>99</id>
                <Level>
                    <id>100</id>                    
                </Level>
            </Type>
        </Flowers>
    </Garden>

I want to use xmltodict to convert this xml to a dictionary and that is pretty simple to do. But there is a slight modification that I would like to do.

I would like to have my json be changed to something like this.

{
    "Garden": {
        "id": "97",
        "state": "0",
        "Flowers": {
            "id": "98",
            "state": "0",
            "Type": {
                "id": "99",
                "state": "0",
                "Level": {
                    "id": "100",
                    "state": "0"                

                }
            }
        }
    }
}

I want to be able to add a default "state": "0" for all the levels. I am really confused on how to do that. Any help would be really appreciated.

This is what I have as of now:

with open("gardenlist.xml", 'r') as file:
    xmlString = file.read() 
print(xmlString)     
jsonString = json.dumps(xmltodict.parse(xmlString), indent=4)

This just prints the json but without the "state": "0" values.

So you want code that will traverse all levels of a nested dictionary, adding `state` to each dictionary that doesn't already have one? What part of this are you having trouble with? — Scott Hunter, Jul 22 '19 at 19:08
@ScottHunter I am having trouble with adding the state to the dictionary. I have update the question with what I have. — Laura Smith, Jul 22 '19 at 19:11
Is it possible for `state` to already be defined in places? If so, what should be done about them? — Scott Hunter, Jul 22 '19 at 19:14
@ScottHunter My code is not adding the state yet. The state is not defined and has to be given a default of "state":"0". This is the part that I am not sure on how to do. There was a typo, its all "state". — Laura Smith, Jul 22 '19 at 19:18
@LauraSmith, why `"status": "0"` , not `state` (on `"Flowers"` level)? — RomanPerekhrest, Jul 22 '19 at 19:19
@ScottHunter: Just write your own parser (see my answer ;-)) — Jan, Jul 24 '19 at 18:01

score 3 · Answer 1 · answered Jul 24 '19 at 18:00

Imo this is a very good opportunity to have your own parser:

from parsimonious.grammar import Grammar
from parsimonious.nodes import NodeVisitor, RegexNode

xml = """
<?xml version="1.0" encoding="UTF-8" ?>
    <Garden>
        <id>97</id>
        <Flowers>
            <id>98</id>
            <Type>
                <id>99</id>
                <Level>
                    <id>100</id>                    
                </Level>
            </Type>
        </Flowers>
    </Garden>
"""

class XMLVisitor(NodeVisitor):
    grammar = Grammar(
        r"""
        program     = root expr+
        expr        = opentag list closetag
        item        = (opentag notpar closetag) / expr
        list        = item+

        root        = ws? lpar "?xml" notpar rpar
        opentag     = ws? lpar word rpar ws?
        closetag    = lpar slash word rpar ws?

        lpar        = "<"
        rpar        = ">"
        notpar      = ~"[^<>]+"
        slash       = "/"

        word        = ~"\w+"
        ws          = ~"\s+"
        """
    )

    def generic_visit(self, node, visited_children):
        return visited_children or node

    def visit_opentag(self, node, visited_children):
        ws, lpar, word, *_ = visited_children
        return word.text

    def visit_closetag(self, node, visited_children):
        lpar, slash, word, *_ = visited_children
        return word.text

    def visit_notpar(self, node, visited_children):
        return node.text

    def visit_item(self, node, visited_children):
        if len(visited_children[0]) == 3:
            # first branch
            opentag, content, *_= visited_children[0]
            return (opentag, content)
        else:
            return visited_children[0]

    def visit_expr(self, node, visited_children):
        tag, lst, *_ = visited_children
        return (tag, lst)

    def visit_program(self, node, visited_children):
        root, content = visited_children
        return self.__makeDict__(content)

    def __makeDict__(self, struct, level = 0):
        d = {}
        for item in struct:
            key, value = item
            if isinstance(value, list):
                value = self.__makeDict__(value)
            d[key] = value
            d["state"] = 0
        return d

visitor = XMLVisitor()
output = visitor.parse(xml)

print(output)

This easy to understand snippet correctly yields

{'Garden': {'id': '97', 'state': 0, 'Flowers': {'id': '98', 'state': 0, 'Type': {'id': '99', 'state': 0, 'Level': {'id': '100', 'state': 0}}}}, 'state': 0}

@LauraSmith: You're welcome. However, better go for an easier to understand answer. — Jan, Jul 24 '19 at 18:05

score 2 · Accepted Answer · answered Jul 22 '19 at 20:00

I would say that the right way is to prepare the needed XML structure - then just convert it to dict and json string:

Complex approach:

import xml.etree.ElementTree as ET
import xmltodict
import json

tree = ET.parse('gardenlist.xml')
root = tree.getroot()

state_el = ET.Element('state')    # prepare `state` node
state_el.text = '0'
root.insert(1, state_el)

def add_state(root, el_to_insert):
    for el in root:
        if len(list(el)):    # check if element has child nodes
            el.insert(1, el_to_insert)
            add_state(el, el_to_insert)


add_state(root, state_el)
json_str = json.dumps(xmltodict.parse(ET.tostring(root, encoding="unicode")), indent=4)
print(json_str)

The actual output:

{
    "Garden": {
        "id": "97",
        "state": "0",
        "Flowers": {
            "id": "98",
            "state": "0",
            "Type": {
                "id": "99",
                "state": "0",
                "Level": {
                    "id": "100",
                    "state": "0"
                }
            }
        }
    }
}

@LauraSmith, I have used the exact input xml you have posted, and it works fine — RomanPerekhrest, Jul 22 '19 at 20:37
Imo the best way is to become even more complicated and write your own parser (see my answer ;-)) — Jan, Jul 24 '19 at 18:02

Akaisteph7 · Answer 3 · 2019-07-22T20:55:37.613

1

You can do this recursively after getting your dictionary. You can check which values also contain dictionaries and add target to those as well. See:

import collections 

def addAttr(target):
    target["state"] = "0"
    for key in target:
        if isinstance(target[key], collections.OrderedDict):
            addAttr(target[key])

d1 = xmltodict.parse(xmlString)
addAttr(d1["Garden"])

edited Jul 22 '19 at 20:55

answered Jul 22 '19 at 19:24

Akaisteph7

5,034
2
20
43

Can the xmltodict take in a file name as arguement? – Laura Smith Jul 22 '19 at 19:49
@LauraSmith I do not think that's possible. Feel free to accept this answer if it answered your question. – Akaisteph7 Jul 22 '19 at 20:00
I see that there is extra "status" associated with the Garden node. Please do verify it. – Laura Smith Jul 22 '19 at 20:24
@LauraSmith changed it. – Akaisteph7 Jul 22 '19 at 20:55

How to convert an xml to a dictionary with some modifications?

3 Answers3