1

PROBLEM: Convert filter(None, Dictionary) to a dictionary again, while filtering empty keys.

I am trying to process some laboratory data using Pandas DataFrame. I am grouping this data, taking into account the first two columns, into groups by using dictionaries; see the following code:

d = {'col1': [1, 1, 1, 2], 'col2': [1, 2, 2, 2], 'col3': [1, 2 ,3, 4], 'col4': [2,1,1,1,], 'col5': [2,1,0,0], 'col6': [2,1,3,1]}
df = pd.DataFrame(data=d)
all_groups = {}
for i in df.col1.unique():
    for j in df.col2.unique():
        all_groups[f'{i}-{j}'] = df[(df.col1 == i) & (df.col2 == j)]

Then I would like to clean all the empty keys by using the function filter: filter(None, all_groups) which appears to be as an answer in the following question:

[how to delete empty dict inside list of dictionary?

In the documentation it says:

Return an iterator yielding those items of iterable for which function(item) is true. If function is None, return the items that are true.

I need to convert this returned sequence to dictionary again but I do not know how because it is simply a filter object.

EDIT 1: I have tried with lambda functions:

 hello = dict(filter(lambda x: len(x) != 0, all_groups))

ValueError: dictionary update sequence element #0 has length 12; 2 is required
Nankin
  • 45
  • 7

2 Answers2

2

in your specific case you may get empty lists as value, to filter this you could do:

  1. by using a list comprehension (this will create a new dict)
    dictionary = {k:v for k, v in dictionary.items() if v}
  1. by using a for loop and removing the keys with empty lists as value(in-place):
    for k, v in dictionary.items():
        if not v:
            dictionary.pop(k)
  1. using filter built-in method (this will create a new dict)
    dictionary = dict(filter(lambda x: x[1], dictionary.items()))

if you have a dict with values that are pd.DataFrame you can filter the empty data frame like:

dict(filter(lambda x: not x[1].empty, all_groups.items()))

output:

{'1-1':    col1  col2  col3  col4  col5  col6
 0     1     1     1     2     2     2,
 '1-2':    col1  col2  col3  col4  col5  col6
 1     1     2     2     1     1     1
 2     1     2     3     1     0     3,
 '2-2':    col1  col2  col3  col4  col5  col6
 3     2     2     4     1     0     1}
oligofren
  • 20,744
  • 16
  • 93
  • 180
kederrac
  • 16,819
  • 6
  • 32
  • 55
  • I was specially interested on the third option. But when I tried the lambda function it did not work for me. By the way, with your answer It says that the truth value of a Dataframe is ambiguous. Now I could make it work by using: dict(filter(lambda x: len(x[1]) != 0, dictionary.items())) (in case you want to modify your answer). Thank you very much!! – Nankin Mar 29 '20 at 21:59
  • @Nankin you can check now my answer if this answer solvea your issue you can upvote or accept, thx! – kederrac Mar 29 '20 at 22:27
0

Another possible answer to this question is, by using lambda function:

hello = dict(filter(lambda x: len(x[1]) != 0, all_groups.items()))
Nankin
  • 45
  • 7