Grouping Dictionary Fields Together Python

Question

So I have this dictionary as an output from different queries:

[{"qty": 12}, {"qty": 12}, {"qty": 12}, {"qty": 12}, {"qty": 12},
{"qty": 12}, {"fail": 0, "pass": 12}, {"fail": 0, "pass": 12},
{"fail": 1}, {"pass": 11}, {"fail": 1}, {"pass": 11}, {"fail": 1},
{"pass": 11}, {"fail": 2}, {"pass": 10}]

the reason for that is because each of the 'qty', 'pass', and 'failed' has different queries and appended to an array.

is there any way I could group it all together and put it to this form? depending on their indexes?

[{"qty": 12, "fail": 0, "pass": 12}, {"qty": 12, "fail": 0, "pass":
12}, {"qty": 12, "fail": 1, "pass": 11}, {"qty": 12, "fail": 1,
"pass": 11}, {"qty": 12, "fail": 1, "pass": 11}, {"qty": 12, "fail":
2, "pass": 10}]

Thank you so much in advance.

I'm a bit confused ... What is the structure you are starting with and what do you want to end up with? — Rushabh Mehta, Jul 05 '18 at 02:37
Do you want to scan through the list, find the first `qty`, the first `pass` and the first `fail` and group them all together into one element (and so on)? — UltraInstinct, Jul 05 '18 at 02:39
@RushabhMehta, i have the first dictionary is the current form of dictionary that I have right now, the second dictionary is what I need as an output .. — Mrii Rya, Jul 05 '18 at 02:42
@UltraInstinct, Yes exactly, [{"qty": 12, "fail": 0, "pass": 12}, {"qty": 12, "fail": 0, "pass": 12}, {"qty": 12, "fail": 1, "pass": 11}, {"qty": 12, "fail": 1, "pass": 11}, {"qty": 12, "fail": 1, "pass": 11}, {"qty": 12, "fail": 2, "pass": 10}] -- is there anyway I could group it like this? — Mrii Rya, Jul 05 '18 at 02:43
What if the number of dicts with 'qty' as a key is not the same as the others. What sort of behavior do you want? — Rushabh Mehta, Jul 05 '18 at 02:59

Sean_Syue · Answer 1 · 2018-07-05T04:37:34.607

UPDATE

Thanks for UltraInstinct and Mankind_008 reminder, a more simplified and intrinsic answer is addressed below:

lst = [{"qty": 12}, {"qty": 12}, {"qty": 12}, {"qty": 12}, {"qty": 12}, {"qty": 12}, {"fail": 0, "pass": 12},
       {"fail": 0, "pass": 12}, {"fail": 1}, {"pass": 11}, {"fail": 1}, {"pass": 11}, {"fail": 1}, {"pass": 11},
       {"fail": 2}, {"pass": 10}]

# separate dictionaries with different keys
# dictionaries containing both "fail" and "pass" keys will be split
# and fitted into "fail_group" and "pass_group" respectively
qty_group = ({key: _} for dic in lst for key, _ in dic.items() if key == "qty")
fail_group = ({key: _} for dic in lst for key, _ in dic.items() if key == "fail")
pass_group = ({key: _} for dic in lst for key, _ in dic.items() if key == "pass")

# merge dictionaries with keys "qty", "fail" and "pass" item-wisely.
# and print result 
print(list({**x, **y, **z} for (x, y, z) in zip(qty_group, fail_group, pass_group)))

Note that {**x, **y, **z} only work on python >= 3.5, which was introduced in PEP 448. For python 2 or python <3.5, you have to define your custom function to do the same thing as {**x, **y, **z} (more details are discussed in this thread):

def merge_three_dicts(x, y, z):
    m = x.copy()
    n = y.copy()
    z.update(x)
    z.update(y)
    return z

So in this scenario, the last line of the code should be:

print(list(merge_three_dicts(x, y, z) for (x, y, z) in zip(qty_group, fail_group, pass_group)))

Both the methods mentioned above will give you the result:

[{'qty': 12, 'fail': 0, 'pass': 12}, {'qty': 12, 'fail': 0, 'pass': 12}, {'qty': 12, 'fail': 1, 'pass': 11}, {'qty': 12, 'fail': 1, 'pass': 11}, {'qty': 12, 'fail': 1, 'pass': 11}, {'qty': 12, 'fail': 2, 'pass': 10}]

You can actually zip all the three lists together in one shot instead of doing 2 zips. +1 — UltraInstinct, Jul 05 '18 at 03:22
yeah same + 1, All in one go: using something like `qty_list = [{key:value} for dict in dict_list for key,value in dict.items() if key == "qty"]` for `qty`, and similar for `fail`, `pass` and then `[{**x, **y, **z} for (x,y,z) in zip(qty_list,fail_list,pass_list)]` — Mankind_008, Jul 05 '18 at 03:50

Pruthvi Kumar · Answer 2 · 2018-07-05T03:11:09.710

2

Instead of appending to one list from your queries, maintain separate lists so you could get a handle on indices for separate lists.

eg:

list1 = [{"qty": 12}, {"qty": 12},]
list2 = [{"fail": 0, "pass": 12}, {"fail": 0, "pass": 12},]

map(lambda (ix, el): el.update(list2[ix]), enumerate(list1))

Now list1 will contain [{"qty": 12,"fail": 0, "pass": 12},... ]

edited Jul 05 '18 at 03:11

answered Jul 05 '18 at 03:05

Pruthvi Kumar

868
1
7
15

ababuji · Accepted Answer · 2018-07-05T03:08:45.283

This is your list of dictionaries. Some have [fail, pass] as keys, some just have [fail] or just [pass] as keys. Don't know if this is intended.

dictList = [{"qty": 12}, {"qty": 12}, {"qty": 12}, {"qty": 12}, {"qty": 12},
{"qty": 12}, {"fail": 0, "pass": 12}, {"fail": 0, "pass": 12},
{"fail": 1}, {"pass": 11}, {"fail": 1}, {"pass": 11}, {"fail": 1},
{"pass": 11}, {"fail": 2}, {"pass": 10}]

My code starts here

failpassList = [] #To keep track of `["fail", "pass"]`
qtyList = [] #To keep track of `["qty"]`

tempDict = {}
checkNext = 0

for index in range(0, len(dictList)):

    #This is the case when, I've seen a key called `fail`, 
    #and now I'm seeing a `pass` right after `fail`, so I will 
    #bring `fail` and `pass` together as keys of a single dictionary.
    #Once this is done, it to `failpassList`

    if checkNext == 1:
        if list(dictList[index].keys()) == ['pass']:
            tempDict.update(dictList[index])
            failpassList.append(tempDict)
            tempDict = {}
            checkNext = 0


    #If the key is `['fail', 'pass']`, then it is correctly 
    #structured, I can append it to `failpassList`.

    elif list(dictList[index].keys()) == ['fail', 'pass']:

        failpassList.append(dictList[index])

    #If the key is `fail` alone, then wait for the next `pass`.
    #I have done this by incrementing a variable called `checkNext`

    elif list(dictList[index].keys()) == ['fail']:

        checkNext += 1
        tempDict = dictList[index]

    #If the key is `qty` put it in a separate list
    else:
        qtyList.append(dictList[index])

Since, qtyList and failpassList will be of the same length, I traverse through one of them, and update the dictionary correspondingly.

for i in range(0, len(qtyList)):

    qtyList[i].update(failpassList[i])

print(qtyList)

will yield:

[{'qty': 12, 'fail': 0, 'pass': 12}, {'qty': 12, 'fail': 0, 'pass': 12}, 
{'qty': 12, 'fail': 1, 'pass': 11}, {'qty': 12, 'fail': 1, 'pass': 11}, 
{'qty': 12, 'fail': 1, 'pass': 11}, {'qty': 12, 'fail': 2, 'pass': 10}]

T. Pieper · Answer 4 · 2020-03-23T13:55:09.040

So the task is to join the dictionaries while keeping the given order.

Assuming some structure like the one below:

query_output = [{'qty': 12}, {'qty': 12}, {'qty': 12}, {'qty': 12}, 
    {'qty': 12}, {'qty': 12}, {'fail': 0, 'pass': 12}, {'fail': 0, 'pass': 12},
    {'fail': 1}, {'pass': 11}, {'fail': 1}, {'pass': 11}, {'fail': 1}, 
    {'pass': 11}, {'fail': 2}, {'pass': 10}]

I would do:

groups = {'qty': [], 'fail': [], 'pass': []}
for d in query_output:
    for k, v in d.items():
        groups[k].append(v)
queries = zip(groups['qty'], groups['fail'], groups['pass'])
result = [{'qty': x, 'fail': y, 'pass': z} for x, y, z in queries]

Grouping Dictionary Fields Together Python

4 Answers4