Python add dict contents to list of dicts

Question

I'm receiving orders as JSON where multiple order-items are added as a list like so:

SAMPLE_ORDER

{
    "importDate": "2020-03-18T10:03:19.194336",
    "status": "shipped",
    "orderNumber": 123456,
    "orderItem": [
        {
            "sku": 998877,
            "itemName": "Doomsday device",
            "netCost": 8.7,
            "quantity": 1
        },
        {
            "sku": 665544,
            "itemName": "Fing longerer",
            "netCost": 99.9,
            "quantity": 1
        }
    ],
    "addressData": {
        "recipientCompany": "Planet Express, Inc",
        "recipientName": "Farnsworth",
        "recipientFirstName": "Hubert",
        "recipientStreet": "72nd Street",
        "recipientHouseNumber": null,
        "recipientAnnex": null,
        "recipientZip": "NNY 10023",
        "recipientCity": "New New York",
        "recipientCountry": "USA",
        "recipientEmail": "hubert@farnsworth.com",
        "recipientPhone": "+123 445-566-7789",
        "recipientMobilePhone": "+123 444 555 666 777"
    }
}

I need to "convert" those JSON-orders into CSV where I create a new row for each ordered item. Above sample would result in a CSV with two rows. This works fine when adding all data the "manual" way - meaning adding address-data like "recipientCompany": order_payload["addressData"]["recipientCompany"].

What I would like to do is adding the whole dictionary order_payload["addressData"] w/o adding each field manually.

I've tried using extend but I only add the keys to the dict and I don't know how to ust list/dict-comprehension "inside" my orders.append()

I've tried to add the key-value-pairs by using something like

(key: value for (key, value) in order_payload["externalReferences"])

but that didn't work either.

I'm sure there is a (very) easy way but I didn't find an answer that helped me in this regard

# order_payload is just a 'json.loads' of the above order
def main(order_payload):
    orders = []
    for order_item in order_payload["orderItem"]:
        orders.append({
            "orderDate": order_payload["importDate"],
            "sku": order_item["sku"],
            "itemName": order_item["itemName"]
            # TODO dynamically add contents of order_payload["addressData"]
        })

Thanks in advance

What fields/cols are you looking for in the output csv, `pandas.json_normalize()` followed by `to_csv()` would be an easy way to create the output you want. — AChampion, Apr 14 '20 at 17:48

Alexandre B. · Accepted Answer · 2020-04-14T17:52:13.153

You can simply merge the dict with the addressData sub-dict. This discussion explains how to perform dict merge.

Code:

for order_item in order_payload["orderItem"]:
    new_dict = {
        **{
            "orderDate": order_payload["importDate"],
            "sku": order_item["sku"],
            "itemName": order_item["itemName"]
            # TODO dynamically add contents of order_payload["addressData"]
        }, 
        **order_payload["addressData"]
    }
    orders.append(new_dict)

If you want the output inline:

orders_2 = [{**{k: v for k, v in data.items() if k not in ["addressData", "orderItem"]},
             **data["addressData"], ** order_item} for order_item in data["orderItem"]]

Full code

data = {
    "importDate": "2020-03-18T10:03:19.194336",
    "status": "shipped",
    "orderNumber": 123456,
    "orderItem": [
        {
            "sku": 998877,
            "itemName": "Doomsday device",
            "netCost": 8.7,
            "quantity": 1
        },
        {
            "sku": 665544,
            "itemName": "Fing longerer",
            "netCost": 99.9,
            "quantity": 1
        }
    ],
    "addressData": {
        "recipientCompany": "Planet Express, Inc",
        "recipientName": "Farnsworth",
        "recipientFirstName": "Hubert",
        "recipientStreet": "72nd Street",
        "recipientHouseNumber": None,
        "recipientAnnex": None,
        "recipientZip": "NNY 10023",
        "recipientCity": "New New York",
        "recipientCountry": "USA",
        "recipientEmail": "hubert@farnsworth.com",
        "recipientPhone": "+123 445-566-7789",
        "recipientMobilePhone": "+123 444 555 666 777"
    }
}

def main(order_payload):
    orders = []
    for order_item in order_payload["orderItem"]:
        new_dict = {
            **{
                "orderDate": order_payload["importDate"],
                "sku": order_item["sku"],
                "itemName": order_item["itemName"]
            }, 
            **order_payload["addressData"]
        }
        orders.append(new_dict)
    return orders

orders_1 = main(data)
orders_2 = [{**{k: v for k, v in data.items() if k not in ["addressData", "orderItem"]},
             **order_item, **data["addressData"]} for order_item in data["orderItem"]]

print(orders_1)
# [
#     {
#         "orderDate": "2020-03-18T10:03:19.194336",
#         "sku": 998877,
#         "itemName": "Doomsday device",
#         "recipientCompany": "Planet Express, Inc",
#         "recipientName": "Farnsworth",
#         "recipientFirstName": "Hubert",
#         "recipientStreet": "72nd Street",
#         "recipientHouseNumber": null,
#         "recipientAnnex": null,
#         "recipientZip": "NNY 10023",
#         "recipientCity": "New New York",
#         "recipientCountry": "USA",
#         "recipientEmail": "hubert@farnsworth.com",
#         "recipientPhone": "+123 445-566-7789",
#         "recipientMobilePhone": "+123 444 555 666 777"
#     },
#     {
#         "orderDate": "2020-03-18T10:03:19.194336",
#         "sku": 665544,
#         "itemName": "Fing longerer",
#         "recipientCompany": "Planet Express, Inc",
#         "recipientName": "Farnsworth",
#         "recipientFirstName": "Hubert",
#         "recipientStreet": "72nd Street",
#         "recipientHouseNumber": null,
#         "recipientAnnex": null,
#         "recipientZip": "NNY 10023",
#         "recipientCity": "New New York",
#         "recipientCountry": "USA",
#         "recipientEmail": "hubert@farnsworth.com",
#         "recipientPhone": "+123 445-566-7789",
#         "recipientMobilePhone": "+123 444 555 666 777"
#     }
# ]
print(orders_2)
# [
#     {
#         "importDate": "2020-03-18T10:03:19.194336",
#         "status": "shipped",
#         "orderNumber": 123456,
#         "sku": 998877,
#         "itemName": "Doomsday device",
#         "netCost": 8.7,
#         "quantity": 1,
#         "recipientCompany": "Planet Express, Inc",
#         "recipientName": "Farnsworth",
#         "recipientFirstName": "Hubert",
#         "recipientStreet": "72nd Street",
#         "recipientHouseNumber": null,
#         "recipientAnnex": null,
#         "recipientZip": "NNY 10023",
#         "recipientCity": "New New York",
#         "recipientCountry": "USA",
#         "recipientEmail": "hubert@farnsworth.com",
#         "recipientPhone": "+123 445-566-7789",
#         "recipientMobilePhone": "+123 444 555 666 777"
#     },
#     {
#         "importDate": "2020-03-18T10:03:19.194336",
#         "status": "shipped",
#         "orderNumber": 123456,
#         "sku": 665544,
#         "itemName": "Fing longerer",
#         "netCost": 99.9,
#         "quantity": 1,
#         "recipientCompany": "Planet Express, Inc",
#         "recipientName": "Farnsworth",
#         "recipientFirstName": "Hubert",
#         "recipientStreet": "72nd Street",
#         "recipientHouseNumber": null,
#         "recipientAnnex": null,
#         "recipientZip": "NNY 10023",
#         "recipientCity": "New New York",
#         "recipientCountry": "USA",
#         "recipientEmail": "hubert@farnsworth.com",
#         "recipientPhone": "+123 445-566-7789",
#         "recipientMobilePhone": "+123 444 555 666 777"
#     }
# ]

Thanks a million! The `**order_payload["addressData"]` is exactly what I was looking for! :-) — Razorfen, Apr 15 '20 at 07:14

Python add dict contents to list of dicts

1 Answers1