Transformation of a data structure to where values can not be available

Question

I try to find a better/nicer/cleaner way to get selected values of a data structure and add them to a new data structure, in this case a class.

I have an object/class where I want to put the selected data into:

class foo:
    def __init__:
        self.name = None
        self.server = None
        self.id = None
        self.foo = None

Now I get the data through json which I dump into a regular data structure (just mentioning in case there is a json hack somehow).

The data could be be in one case

data = {
    'id': '1234',
    'x': {
         'y': 'asdfasdf',
    },
    'a': {
        'b': 'foo'
    }
}

For such a case I can easily assign it in a function via

self.id = data['id'],
self.foo = data['x']['y']
self.name = data['a']['b']

The challenge starts when the data I get doesn't have all the values defined.

In the following case data['x'] isn't defined and data['a']['b'] doesn't exist, either.

data = {
    'id': '2345',
    'a': {
        'X': 'bar'
    }
}

As result, with the second input data I can't use a simple

self.id = data['id'],
self.foo = data['x']['y']
self.name = data['a']['b']

Of course I can do all these test like

if 'x' data  and  'y' in data['x']
    self.foo = data['x']['y']
if 'b' in data['a']: # assuming data['a'] always exists
    self.name = data['a']['b']

Now what I am looking for is a slick, much shorter & nicer to read option to process the input data even if a value isn't defined. Without the requirement to check if a value exist. Assigning it if it exists and ignoring it (or setting it to None) if it doesn't exists.

Something that scales to much more variables and depth as the two values provided in the above example.

One option I thought about was maybe create a mapping and then do a function that does all the checks based on the mapping. This also comes with some own error checking logic. Like:

mappings = {
    'id': 'id',
    'foo': 'x.y.',
    'name': 'a.b'
}

data_input = {
    'id': '2345',
    'a': {
        'X': 'bar'
    }
}

map_function(mappings, data_input, class_object)

# Pseudo code, could contain syntax errors
# Probably also a lot of improvements possible like doing the depth recursively.
# This is just a big picture logic POC
def map_function(mappings, data_input, class_object):
    for mapping in mappings:
        for case in mappings[mapping].split('.'):
            if case in data_input:
                if type(data_input[case]) in ['str', 'int', 'NoneType']):
                    # need to figure out how to use the value here
                    # and don't set it to `mapping`
                    class_object.mapping = data_input[case]
                 else:
                    # go down another layer
                    # A recursion function execution would work best her
                    ...
             else:
                 continue

@darthbith: `get` is great. I didn't know about it. That was my hope to find a simple solution I am not aware off. For a nested structure I have to define the default value such as `{'foo': 'bar'}.get('foo', {}).get('x'')`. There seems to be an edge case, though. When the first value is specifically defined as `None` i get an error, e.g. `{'foo': None}.get('foo'. {}).get('.bar')`. For this case `.get()` might not be a good solution since it would result in a lot of exception handling. Basically each line one try & except? — volker, Nov 05 '18 at 17:03
At some point, if you really don't have any structure to your data (or the structure is really unknown), then yes you're going to have a lot of if/else or try/except statements. I don't really think there's a way around that. BTW, you have a lot of typos in this post in the code, it would be good to clean them up to make sure people focus on the issue that you have rather than the typos. — darthbith, Nov 05 '18 at 19:37

score 1 · Answer 1 · answered Nov 05 '18 at 16:38

Types are essentially dictionaries so you can easily coerce from one to the other. In the answer given to this question, they did this:

class obj(object):
    def __init__(self, d):
        for a, b in d.items():
            if isinstance(b, (list, tuple)):
                setattr(self, a, [obj(x) if isinstance(x, dict) else x for x in b])
            else:
                setattr(self, a, obj(b) if isinstance(b, dict) else b)

You might want to modify it for your specific purpose by adding a mapping in there, for instance:

class obj(object):
    def __init__(self, d, mapping):
        for a, b in d.items():
            if (a not in mapping):
                continue
            if isinstance(b, (list, tuple)):
                setattr(self, mapping[a], [obj(x) if isinstance(x, dict) else x for x in b])
            else:
                setattr(self, mapping[a], obj(b) if isinstance(b, dict) else b)

Once you've done that, just change the __name__ property on the class and return it:

 def Convert(d, mapping, name):

    class obj(object):
        def __init__(self, d, mapping):
            for a, b in d.items():
                if (a not in mapping):
                    continue
                if isinstance(b, (list, tuple)):
                    setattr(self, mapping[a], [obj(x) if isinstance(x, dict) else x for x in b])
                else:
                    setattr(self, mapping[a], obj(b) if isinstance(b, dict) else b)

    result = obj(d, mapping)
    obj.__name__ = name
    return result

Note that this solution abuses duck typing pretty heavily. This might yield some strange behavior so use with caution.

Transformation of a data structure to where values can not be available

1 Answers1