How to populate optional fields in a dictionary comprehension?

Question

I'm working on some code that processes a json database with very detailed information into a simpler format. It copies some of the fields and reserializes others into a new json file.

I'm currently using a dictionary comprehension like this MVCE:

converted_data = {
    raw_item['name']: {
        'state': raw_item['field_a'],
        'variations': [variant for variant in raw_item['field_b']['variations']]
    } for raw_item in json.loads(my_file.read())
}

An example file (not the actual data being used) is this:

[
    {
        "name": "Object A",
        "field_a": "foo",
        "field_b": {
            "bar": "baz",
            "variants": [
                "foo",
                "bar",
                "baz"
            ]
        }
    },
    {
        "name": "Object B",
        "field_a": "foo",
        "field_b": {
            "bar": "baz",
        }
    }
]

The challenge is that not all items contain variations. I see two potential solutions:

Use an if statement to conditionally apply the variations field into the dictionary.
Include an empty variations field for all items and fill it if the raw item contains variations.

I'll probably settle on the 2nd solution. However, is there a way to conditionally include a particular field within a dictionary comprehension?

Edit: In other words, is approach 1 possible inside a dictionary comprehension?

An example of the desired output (using a dictionary comprehension) would be as follows:

{
    "Object A": {
        "state": "foo",
        "variants": ["foo", "bar", "baz"]
    },
    "Object B": {
        "state": "foo"
    }
}

I've found some other questions that change the entries conditionally or filter the entries, but these don't unconditionally create an item where a particular field (in the item) is conditionally absent.

@kaya3 Alrighty, since it was an example instead of the actual code and data, I hadn't included sample data; it didn't exist. I've now included some sample data for the example and removed the `lookup_table` call I'd included to roughly mirror the behaviour of the actual code (and prevent the inevitable questions about the purpose of that line). — Aaron3468, Dec 13 '19 at 00:52

Grismar · Accepted Answer · 2019-12-13T01:29:07.513

3

I'm not sure you realise you can use the if inside an assignment, which seems like a very clean way to solve it to me:

converted_data = {
    raw_item['name']: {
        'state': raw_item['field_a'],
        'variants': [] if 'variants' not in raw_item['field_b'] else
            [str(variant) for variant in raw_item['field_b']['variants']]
    } for raw_item in example
}

(Note: using str() instead of undefined function that was given in initial example)

After clarification of the question, here's an alternate solution that adds a different dictionary (missing the empty 'variations' key if there is none:

converted_data = {
    raw_item['name']: {
        'state': raw_item['field_a'],
        'variants': [str(variant) for variant in raw_item['field_b']['variants']]
    } if 'variants' in raw_item['field_b'] else {
        'state': raw_item['field_a'],
    } for raw_item in example
}

If the question actually is: can a key/value pair in a dictionary literal be optional (which would solve your problem) then the answer is simply "no". But the above achieves the same for this simple example.

If the real life situation is more complicated, simply construct the dictionary as in the first solution given here and then use del(dictionary['key']) to remove any added keys that have a None or [] value after construction.

For example, after the first example, converted_data could be cleaned up with:

for item in converted_data.values:
    if not item['variants']:
        del(item['variants'])

edited Dec 13 '19 at 01:29

answered Dec 13 '19 at 00:16

Grismar

27,561
4
31
54

I've updated the question. Hopefully the changes make it more clear what the question was; the `variations` field would preferably not exist if it hadn't existed in the object being converted. I've already completed solution 2 (which you've suggested in your current code), but I'm way more curious if solution 1, the subject of the question, exists. – Aaron3468 Dec 13 '19 at 00:53
clean the data afterwards after using @Grismar method above. Eg. `'variations': False if 'field_b' not in raw_item else "some data"`. Then check if the value equates to true, otherwise delete the key/val. – Ari Dec 13 '19 at 01:00
I've added additional code that result in what you're asking - it produces the desired result given the example you provided. – Grismar Dec 13 '19 at 01:17
@Grismar Good work, I've selected your question as the answer. I was quite surprised that for this case python doesn't have a non-procedural way to do this! – Aaron3468 Dec 13 '19 at 02:08
@Aaron3468 I see what you mean, but it probably has something to do with the trade off between allowing that level of flexibility and getting good performance out of dictionary creation with literals. If this problem happens more often for you, you could consider just writing a nice and clean `dict_vacuum()` function or something similar to clean out empty / unused items, to wrap around the comprehension, but I expect it will be too specific a problem. – Grismar Dec 13 '19 at 03:55

score 1 · Answer 2 · answered Dec 13 '19 at 00:12

You could pass the process out to a function?

def check_age(age):
    return age >= 18

my_dic = {
    "age": 25,
    "allowed_to_drink": check_age(25)
}

You end up with the value as the result of the function call

{'age': 25, 'allowed_to_drink': True}

How you would implement this I don't know, but some food for thought.

How to populate optional fields in a dictionary comprehension?

2 Answers2