3

We have a config.yaml file like this:

darwin:
  installer:
    title: "%(product_name)s %(version)s"
    filename: "%(brand_name)s-%(version)s"

and a function to format it:

def format_context(config):
    return {
        "company_name": config['company_name'],
        "product_name": config['product_name'],
        "brand_name": config['brand_name'],
        "title": config['darwin']['installer']['title'],
        "filename": config['darwin']['installer']['filename'],
    }

The goal here is we can input the value as a formatted string. Now I need to turn the dictionary return by format_context into variables.

The first try is use locals():

context = format_context(config)
for k, v in context.iteritems():
    locals()[k] = str(v) % context

But maybe due to the order, I sometimes got a KeyError error. And moreover, from the Python doc:

Note The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.

So, I switched to use exec:

context = format_context(config)
for k, v in context.iteritems():
    exec("%s = '%s'" % (k, str(v) % context))

It works but I'm wondering is this a good way?


Please let me clarify why I'm going to create variables from that dict. I have a function to parse this config.yaml:

class BrandConfiguration(object):
    """
    A brand configuration (directory)
    """

    def __init__(self, directory):
        self.dirname = directory

    @property
    def config(self):
        """
        return configuration for a single brand
        """
        with open(os.path.join(self.dirname, "config.yaml")) as fh:
            return yaml.load(fh)

Then in one class, I defined some variables:

-        brand_config = self.brand_config_instance.config
-        binary_name = brand_config['binary_name']
-        major_version = brand_config['version']['major']
-        minor_version = brand_config['version']['minor']
-        patch_version = brand_config['version']['patch']

In another class (or another Python file), I need to do the same thing:

 -    brand_name, binary_name = config['brand_name'], config['binary_name']
 -    identifiers = [binary_name] + brand_name.split('.')
 -    identifiers.reverse()
 -    identifier = '.'.join(identifiers)
 -    major_version = config['version']['major']
 -    minor_version = config['version']['minor']
 -    patch_version = config['version']['patch']
 -    version = '.'.join(
 -        (
 -            str(major_version),
 -            str(minor_version),
 -            str(patch_version),
 -            build_number
 -        )
 -    )

Since I don't want to duplicate the code, I'm trying store it all in a dictionary and convert it into variables.


Where/how are you trying to use the values from the dictionary returned by format_context?

Assumming that in config.yaml, you have something like this:

version:
  major: 1
  minor: 0
  patch: 0

When adding metadata for Windows, instead of creating some variables:

-    brand_name, binary_name = config['brand_name'], config['binary_name']
-    identifiers = [binary_name] + brand_name.split('.')
-    identifiers.reverse()
-    identifier = '.'.join(identifiers)
-    major_version = config['version']['major']
-    minor_version = config['version']['minor']
-    patch_version = config['version']['patch']
-    version = '.'.join(
-        (
-            str(major_version),
-            str(minor_version),
-            str(patch_version),
-            build_number
-        )
-    )

now I can use it directly:

    json_data['FixedFileInfo']['FileVersion']['Major'] = major_version
    json_data['FixedFileInfo']['FileVersion']['Minor'] = minor_version
    json_data['FixedFileInfo']['FileVersion']['Patch'] = patch_version
    json_data['FixedFileInfo']['FileVersion']['Build'] = build_number

    json_data['FixedFileInfo']['ProductVersion'] = \
        json_data['FixedFileInfo']['FileVersion']

    json_data['StringFileInfo']['CompanyName'] = company_name
    json_data['StringFileInfo']['FileDescription'] = service_description
    json_data['StringFileInfo']['LegalCopyright'] = legal_copyright
    json_data['StringFileInfo']['ProductName'] = product_name
    json_data['StringFileInfo']['ProductVersion'] = '.'.join(
        (
            str(major_version),
            str(minor_version),
            str(patch_version),
            self.target.build_number
        )
    )
quanta
  • 3,960
  • 4
  • 40
  • 75
  • 4
    I don't understand why you're trying to create variables, or what the YAML document has to do with the rest of your question. If you want to replace the `%(...)s` expression in it, just set `formatted_yaml = raw_yaml % format_context(config)`. – larsks Nov 05 '16 at 03:35
  • I need to set some common variables from that `config.yaml` file, and I don't want to duplicate the code. I updated my original quesiton to clarify. Is it more clear now? – quanta Nov 05 '16 at 04:01
  • I'm still confused, because your `BrandConfiguration` class doesn't seem to be doing any templating. Where/how are you trying to use the values from the dictionary returned by `format_context`? I strongly suspect that using `locals()` is the wrong solution here, but since I'm not clear what you're trying to accomplish I'm not sure I can offer a better solution. – larsks Nov 05 '16 at 04:04
  • @larsks I updated my question again. Thanks. – quanta Nov 05 '16 at 04:18
  • "I need to turn the dictionary return by `format_context` into variables." --- I still don't see why this is so. It looks like a dictionary is already the ideal storage for your data, especially if it all gets used in the same place(s) like that. At most, you might want separate dictionaries for related clumps of data like `'FileVersion'`. Also, I don't see any duplicated code. – Kevin J. Chase Nov 05 '16 at 04:30
  • _It looks like a dictionary is already the ideal storage for your data, especially if it all gets used in the same place(s) like that._ --> No, and it's a different places. _At most, you might want separate dictionaries for related clumps of data like 'FileVersion'_ --> No, this should be the same for all OSes. _Also, I don't see any duplicated code._ --> after parsing the yaml file, I need to set some variables (major_version, minor_version, patch_version, ... and so on) to do something for Windows. Then I need to do the same (in another file) for other OSes. – quanta Nov 05 '16 at 04:37
  • "Then I need to do the same (in another file) for other OSes." --- In another _Python_ file? If the code's the same, just use one function in a common utilities module, or one method in a base class inherited by OS-specific subclasses. If the data structure is "the same for all OSes", I don't see why you'd need to repeat the code that interprets it. – Kevin J. Chase Nov 05 '16 at 05:36
  • To return to your original question: Why are you trying to create local variables in the first place? You still haven't explained _why_ dictionaries aren't good enough. They are designed to do exactly this job, and they avoid all kinds of ugly surprises --- like strings that are valid `dict` keys but not valid Python variable names (`'123abc'`), or strings that happen to reassign important builtins (`'list'` or `'str'`). – Kevin J. Chase Nov 05 '16 at 05:40
  • @KevinJ.Chase _In another Python file?_ --> Yes. _If the code's the same, just use one function in a common utilities module, or one method in a base class inherited by OS-specific subclasses._ --> It's just the code that set the variables is the same, not the logic after that. _If the data structure is "the same for all OSes", I don't see why you'd need to repeat the code that interprets it._ --> As I said, it's just the code that set some common variables is the same. Please look at my edited original question, I think it's more clear now. – quanta Nov 05 '16 at 07:17
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/127423/discussion-between-quanta-and-kevin-j-chase). – quanta Nov 05 '16 at 07:17
  • 1
    If you need these variables in a function just pass the dictionary in via `**kwargs` – hpaulj Nov 05 '16 at 15:15

1 Answers1

0

argparse users sometimes ask about converting the args attributes into a globals or locals, e.g.

Python argparse parse_args into global namespace (or a reason this is a bad idea)

python argparse, how to refer args by their name

argparse returns the parsed arguments as a Namespace object, which is defined with this code:

In [245]: argparse.Namespace??
Init signature: argparse.Namespace(**kwargs)
Source:        
class Namespace(_AttributeHolder):
    """Simple object for storing attributes.

    Implements equality by attribute names and values, and provides a simple
    string representation.
    """

    def __init__(self, **kwargs):
        for name in kwargs:
            setattr(self, name, kwargs[name])

    def __eq__(self, other):
        if not isinstance(other, Namespace):
            return NotImplemented
        return vars(self) == vars(other)

    def __contains__(self, key):
        return key in self.__dict__
File:           /usr/lib/python3.5/argparse.py
Type:           type

Such an object can be created from a dictionary with

In [247]: adict={'a':1,'b':'aname','c':[1,2,3]}
In [248]: ns=argparse.Namespace(**adict)
In [249]: ns
Out[249]: Namespace(a=1, b='aname', c=[1, 2, 3])

And the attributes can be accessed by name:

In [250]: ns.a
Out[250]: 1
In [251]: ns.b
Out[251]: 'aname'
In [252]: ns.c
Out[252]: [1, 2, 3]

This syntax is just like asking for variable a in a module named ns.

It can be easily turned back into a dictionary with vars:

In [253]: vars(ns)
Out[253]: {'a': 1, 'b': 'aname', 'c': [1, 2, 3]}

Note that Namespace sets the attributes with setattr. This is more powerful than the `ns.abc='123' syntax, since it create attributes that aren't valid variable names:

In [254]: setattr(ns,'123','foo')
In [255]: ns
Out[255]: Namespace(123='foo', a=1, b='aname', c=[1, 2, 3])
In [256]: ns.123
  File "<ipython-input-256-f3ac9938f9b1>", line 1
    ns.123
         ^
SyntaxError: invalid syntax

In [257]: getattr(ns,'123')
Out[257]: 'foo'

Some programs, like Ipython, load arguments from a config file(s), and then use argparse to read last-minute arguments from the commandline, giving users several ways of setting options. It is usually more useful to keep those options, regardless of source, collected in one place (a namespace or dictionary) rather than merged into the globals or locals.

Community
  • 1
  • 1
hpaulj
  • 221,503
  • 14
  • 230
  • 353