Deserialize nested fields in marshmallow

Question

I'm consuming an API that returns something like:

{'name': 'foo', 'start': {'date': '2016-06-19', 'time': '18:00'}}

And I want to desearialize it with marshmallow to get only the name and the start date, so the desired result would be the following:

{'name': 'foo', 'date': '2016-06-19'}

But I haven't found any way to get the date, this what I have tried:

from marshmallow import Schema, fields, pprint

event = {'name': 'foo', 'start': {'date': '2016-06-19', 'time': '18:00'}}
class EventSchema(Schema):
    name = fields.Str()
    date = fields.Str(load_from='start.date')


schema = EventSchema()
result = schema.load(event)
pprint(result.data)

score 21 · Answer 1 · edited Feb 27 '19 at 05:42

What you describe can be accomplished by transforming* your input data in a pre-processing* step. While the accepted answer looks like it will do that, Marshmallow has built-in decorators to allow you to accomplish this in a way that I think is even clearer:

from marshmallow import Schema, pre_load, fields, pprint

event = {'name': 'foo', 'start': {'date': '2016-06-19', 'time': '18:00'}}
expected = {'name': 'foo', 'date': '2016-06-19'}


class EventSchema(Schema):
    name = fields.Str()
    # Marshmallow 2
    date = fields.Str(load_from='date')
    # Marshmallow 3
    date = fields.Str(data_key='date')

    @pre_load
    def move_date(self, data):
        """This will alter the data passed to ``load()`` before Marshmallow
        attempts deserialization.
        """
        start = data.pop('start')
        data['date'] = start['date']
        return data

schema = EventSchema()
result = schema.load(event)
pprint(result.data)

assert result.data == expected

* transform and pre-process are terms of art in the domain of object modeling and data processing. I bolded them because knowing these might help folks who read this question successfully Google for answers to related questions.

Excellent answer. This might benefit from a review of the latest documentation, as this may be non-obvious to developers who are not familiar with decorator syntax in python. — dreftymac, Jun 29 '18 at 20:50
Just a heads up, as of version 3.0.0b8 `load_from` and `dump_to` were replaced with the `data_key` parameter: https://marshmallow.readthedocs.io/en/dev/api_reference.html?highlight=load_from#module-marshmallow.fields — jket, Feb 04 '19 at 06:04

Moses Koledoye · Accepted Answer · 2016-09-14T13:44:30.820

You will need to create a NestedSchema for the nested dictionary, and overwrite your parent schema's load method to append the nested field to parent. Specify a only attribute so the Nested field does not fetch all of its items:

class DateTimeSchema(Schema):
    date = fields.Str()
    time = fields.Str()


class EventSchema(Schema):
    name = fields.Str()
    date = fields.Nested(DateTimeSchema, load_from='start', only='date')

    def load(self, *args, special=None):
        _partial = super(EventSchema, self).load(*args)

        # Move special field from Nest to Parent
        if special is not None and special in _partial.data:
            _partial.data[special]  = _partial.data[special].get(special)
        return _partial

And setting up your schema instance like so:

event = {'name': 'foo', 'start': {'date': '2016-06-19', 'time': '18:00'}}

schema, special_field = EventSchema(), 'date'
result = schema.load(event, special=special_field)
pprint(result.data)
# {'name': 'foo', 'date': '2016-06-19'}

You can always fine tune to your taste.

You can also cut the story short with lambdas, http://marshmallow.readthedocs.io/en/latest/custom_fields.html#function-fields, doing something like this: `date = fields.Function(load_from='start', load_only=True, deserialize=lambda start: start['date'])`. — Luka Žitnik, Jul 18 '17 at 13:38

score 3 · Answer 3 · answered Aug 29 '19 at 22:11

3

Marshmallow 3 has Pluck:

class DateTimeSchema(Schema):
    date = fields.Str()
    time = fields.Str()

class EventSchema(Schema):
    name = fields.Str()
    date = fields.Pluck(DateTimeSchema, 'date')

documentation for fields.Pluck()

answered Aug 29 '19 at 22:11

MarredCheese

17,541
8
92
91

3

Isn't `Pluck` the opposite of what @camaya is asking for? I think camaya needs to "pluck" a field during *deserialization*, whereas `Pluck` plucks a field out for *serialization*, if I'm reading the docs right? – Dale May 12 '20 at 05:36

score 1 · Answer 4 · answered Mar 09 '22 at 14:16

Seeing how complex the previous answers are, I want to offer a very simple solution (using fields.Function):

from marshmallow import Schema, fields

class EventSchema(Schema):
    name = fields.Str()
    date = fields.Function(data_key='start',
                           deserialize=lambda start: start['date'])

schema = EventSchema()
event = {'name': 'foo', 'start': {'date': '2016-06-19', 'time': '18:00'}}
result = schema.load(event)
print(result)

The result is as expected: {'name': 'foo', 'date': '2016-06-19'}.

This works on version 3 of marshmallow.

Deserialize nested fields in marshmallow

4 Answers4