1

I have the following dict:

my_dict = {'a': [['a','b','c'], ['a',1,4]], 'b':[[1,2,3], [1],[8,2,2,1]]}

(The original dict is much larger)

I want to go over all values, merge lists in one and remove duplicates, for each key.

I am doing it with the following method:

merged_dicts_list = [{k:list(set(list(chain.from_iterable(v))))} for k,v in my_dict.items()]

But I keep getting the error:

TypeError: unhashable type: 'dict'

Please advise how to flatten/merge the lists and remove the duplicates + convert back to list to avoid this error.

Finally I want to get:

[{'a': ['a','b','c',1,4]}, {'b': [1,2,3,8]}]
SteveS
  • 3,789
  • 5
  • 30
  • 64

3 Answers3

1

this works :

for k, v in my_dict.items():
    my_dict[k] =  [item for sublist in v for item in sublist]

Edit : OOPS my bad ! that did not work, now this does :

for k, v in my_dict.items():
    my_dict[k] = list( dict.fromkeys( [item for sublist in v for item in sublist]) )

Output :

{'a': ['a', 'b', 'c', 1, 4], 'b': [1, 2, 3, 8]}
grymlin
  • 492
  • 1
  • 9
0

Finally I want to get:

[{'a': ['a','b','c',1,4]}, {'b': [1,2,3,8]}]

my_dict = {'a': [['a','b','c'], ['a',1,4]], 'b':[[1,2,3], [1],[8,2,2,1]]}
merged_dicts_list = list(dict())
for k, v in my_dict.items():
    distinct_val = []
    for v_in in v:
        for v_in_in in v_in:
            if v_in_in not in distinct_val:
                distinct_val.append(v_in_in)
    merged_dicts_list.append({k : distinct_val})

print(merged_dicts_list)

Outputs exactly what were you asking for:

[{'a': ['a', 'b', 'c', 1, 4]}, {'b': [1, 2, 3, 8]}]
RobertG
  • 416
  • 1
  • 8
-1

Slightly alternate solution using sum, then converting to set to remove duplicate and finally converting back to list:

>>> {k: list(set(sum(v, []))) for k,v in my_dict.items()}

# Output:
{'a': [1, 4, 'a', 'b', 'c'], 'b': [8, 1, 2, 3]}

But if you want to maintain the order of the values, then conversion to set will not work, you will have to follow alternate approach to remove the duplicates. You can follow How do I remove duplicates from a list, while preserving order? thread for that.

ThePyGuy
  • 17,779
  • 5
  • 18
  • 45
  • ```sum(v, [])``` means sum all the lists into one list? @thepyguy? – SteveS Sep 22 '22 at 12:09
  • 1
    Yes. Its like `x+y+z` where `x,y,z` are list inside a list/iterable. – ThePyGuy Sep 22 '22 at 12:09
  • 1
    I don't know who downvoted but I have voted for it and accepted the answer. Please explain, sum will be faster than itertools? @thepyguy – SteveS Sep 22 '22 at 12:21
  • 2
    Downvoted. 1. The example code of OP works normally on my machine. I think we should ask the OP to clarify the question before giving the answer. 2. `sum` is not a good solution here. Its space-time complexity is higher than `list(chain.from_iterable())` (because each list concatenation uses a non in-place operator). – Mechanic Pig Sep 22 '22 at 12:22
  • @SteveS I'm slightly dubious regarding the efficiency, but I'll try to fix your solution and add that too to answer then you can see which is the faster one. – ThePyGuy Sep 22 '22 at 12:23
  • @SteveS actually your code already works fine – ThePyGuy Sep 22 '22 at 12:24
  • 1
    Also downvoted. Using sum for list concatenation is a bad habit that will eventually burn the programmer in other situations. – Steven Rumbalski Sep 22 '22 at 13:09
  • @StevenRumbalski thanks for your response. I just proposed it as an alternative solution, though I know its not so efficient, but a concise solution without using any other libraries. – ThePyGuy Sep 22 '22 at 13:10
  • 2
    @ThePyGuy If it is for the reason of not using other libraries, it should be two-layer list comprehension, `sum` is not suitable for this. – Mechanic Pig Sep 22 '22 at 15:16
  • Used ```list(chain.from_iterable()``` – SteveS Sep 26 '22 at 17:37