Import dictionary in loop in Python

Question

My question is related to this one. I have a config_file consisting of dictionaries as shown below:

config_1 = {
    'folder': 'raw1',
    'start_date': '2019-07-01'
}
config_2 = {
    'folder': 'raw2',
    'start_date': '2019-08-01'
}
config_3 = {
    'folder': 'raw3',
    'start_date': '2019-09-01'
}

I then have a separate python file that imports each config and does some stuff:

from config_file import config_1 as cfg1
Do some stuff using 'folder' and 'start_date'

from config_file import config_2 as cfg2
Do some stuff using 'folder' and 'start_date'

from config_file import config_2 as cfg3
Do some stuff using 'folder' and 'start_date'

I would like to put this in a loop rather than have it listed 3 times in the python file. How can I do that?

So it seems eval, does not work, as I first commented. But [this](https://stackoverflow.com/questions/6677424/how-do-i-import-variable-packages-in-python-like-using-variable-variables-i) other question has another solution. — MikeMajara, Sep 26 '19 at 16:01

Massifox · Accepted Answer · 2019-09-26T18:02:32.263

If I understand your question correctly, just use importlib. In a nutshell, what in python you write like:

from package import module as alias_mod

in importlib it becomes:

alias_mod = importlib.import_module('module', 'package')

or, equivalentelly:

alias_mod = importlib.import_module('module.package')

for example:

from numpy import random as rm

in importlib:

rm = importlib.import_module('random', 'numpy')

Another interesting thing is this code proposed in this post, which allows you to import not only modules and packages but also directly functions and more:

def import_from(module, name):
    module = __import__(module, fromlist=[name])
    return getattr(module, name)

For your specific case, this code should work:

import importlib

n_conf = 3
for in range(1, n_conf)
    conf = importlib.import_module('config_file.config_'+str(i))
    # todo somethings with conf

However, if I can give you some advice I think the best thing for you is to build a json configuration file and read the file instead of importing modules. It's much more comfortable. For example in your case, you can create a config.json file like this:

{
    "config_1": {
        "folder": "raw1",
        'start_date': '2019-07-01'
    },
    "config_2": {
        'folder': 'raw2',
        'start_date': '2019-08-01'
    },
    "config_3": {
        'folder': 'raw3',
        'start_date': '2019-09-01'
    }
}

Read the json file as follows:

import json
with open('config.json') as json_data_file:
    conf = json.load(json_data_file)

Now you have in memory a simple python dictionary with the configuration settings that interest you:

conf['config_1']
# output: {'folder': 'raw1', 'start_date': '2019-07-01'}

score 1 · Answer 2 · answered Sep 26 '19 at 16:25

You can use inspect module to get all possible imports from config like following.

import config
import inspect

configs = [member[1] for member in inspect.getmembers(config) if 'config_' in member[0]]

configs

And then you can iterate over all configs, is this the behavior you wanted?

You can read more about inspect here

.

Gaurav Bansal · Answer 3 · 2019-10-03T14:03:30.483

0

Based on @MikeMajara's comment, the following solution worked for me:

package = 'config_file'
configs = ['config_1', 'config_2', 'config_3']
for i in configs:
    cfg = getattr(__import__(package, fromlist=[configs]), i)
    Do some stuff using 'folder' and 'start_date'

edited Oct 03 '19 at 14:03

answered Sep 26 '19 at 17:07

Gaurav Bansal

5,221
14
45
91

Import dictionary in loop in Python

3 Answers3