Python ConfigParser interpolation from foreign section

Question

With Python ConfigParser, is it possible to use interpolation across foreign sections? My mind seems to tell me I've seen that it's possible somewhere, but I can't find it when searching.

This example doesn't work, but it's to give an idea of what I'm trying to do.

[section1]
root = /usr

[section2]
root = /usr/local

[section3]
dir1 = $(section1:root)/bin
dir2 = $(section2:root)/bin

Note that I'm using Python 2.4.

score 14 · Accepted Answer · edited Nov 23 '14 at 11:06

In python 3.2 and up this is perfectly valid:

[Common]
home_dir: /Users
library_dir: /Library
system_dir: /System
macports_dir: /opt/local

[Frameworks]
Python: 3.2
path: ${Common:system_dir}/Library/Frameworks/

[Arthur]
nickname: Two Sheds
last_name: Jackson
my_dir: ${Common:home_dir}/twosheds
my_pictures: ${my_dir}/Pictures
python_dir: ${Frameworks:path}/Python/Versions/${Frameworks:Python}

Edit:

I just saw that you are using python 2.4, so no, section interpolation cannot be done in python 2.4. It was introduced in python 3.2 - See section 13.2.5 - ConfigParser Interpolation of values.

class configparser.ExtendedInterpolation

An alternative handler for interpolation which implements a more advanced syntax, used for instance in zc.buildout. Extended interpolation is using ${section:option} to denote a value from a foreign section. Interpolation can span multiple levels. For convenience, if the section: part is omitted, interpolation defaults to the current section (and possibly the default values from the special section). For example, the configuration specified above with basic interpolation, would look like this with extended interpolation:
   [Paths]
   home_dir: /Users
   my_dir: ${home_dir}/lumberjack
   my_pictures: ${my_dir}/Pictures
Values from other sections can be fetched as well:
   [Common]
   home_dir: /Users
   library_dir: /Library
   system_dir: /System
   macports_dir: /opt/local

   [Frameworks]
   Python: 3.2
   path: ${Common:system_dir}/Library/Frameworks/

   [Arthur]
   nickname: Two Sheds
   last_name: Jackson
   my_dir: ${Common:home_dir}/twosheds
   my_pictures: ${my_dir}/Pictures
   python_dir: ${Frameworks:path}/Python/Versions/${Frameworks:Python}

Looks like your syntax is a bit different than mind. You're using ${section:name} instead of what I was using, %(section:name)s. I tried this and didn't work. — user16738, Sep 30 '11 at 02:29
Ah, sorry, I didnt realize you were referring to string interpolation across sections. I believe that came with 3.2 as well, let me double check. — chown, Sep 30 '11 at 02:30
I just found what you have here in the Python 3.2 docs (configparser.ExtendedInterpolation), but it doesn't list when this was added. When I look at the Python 2.4 docs (what I'm using), this isn't shown. I'm guessing it was added somewhere in between 2.4 and 3.2. — user16738, Sep 30 '11 at 02:33
Yea, I checked each versions docs from 2.4 to 3.2 for ConfigParser and the first one to have the ExtendedInterpolation class was 3.2. — chown, Sep 30 '11 at 02:40
They should probably add "New in Python 3.2" to the docs. It comes in handy. — user16738, Sep 30 '11 at 02:59

score 2 · Answer 2 · answered Jul 01 '15 at 08:23

2

You do have access to the special-case [DEFAULT] section. Values defined here can be accessed via interpolation from other sections even for older versions of Python.

answered Jul 01 '15 at 08:23

Papadeltasierra

233
1
9

Could you elaborate a little more? and possibly an example please. Or even the source reference. – Hamed Jul 15 '15 at 14:59
Hamed, take a look here. https://docs.python.org/2/library/configparser.html. Basically (and you can download and read the code) if you try to read the 'value' for a 'name' from a specific [section], but there is no 'name=value' in that [section], then the config parser looks in the [DEFAULT] section for a 'name=value' and returns that if present. – Papadeltasierra Sep 24 '15 at 12:21

score 0 · Answer 3 · answered Nov 23 '14 at 11:56

If you're stuck with python 2.7 and you need to do cross-section interpolation it is easy enough to do this by hand using regexps.

Here is the code:

INTERPOLATION_RE = re.compile(r"\$\{(?:(?P<section>[^:]+):)?(?P<key>[^}]+)\}")

def load_something_from_cp(cp, section="section"):
    result = []
    def interpolate_func(match):
        d = match.groupdict()
        section = d.get('section', section)
        key = d.get('key')
        return cp.get(section, key)
    for k, v in cp.items(section):
        v = re.sub(INTERPOLATION_RE, interpolate_func, v)
        result.append(
            (v, k)
        )
    return result

Caveeats:

There is no recursion in interpolation
When parsing many sections, youll need to somehow guess current section.

score 0 · Answer 4 · answered Mar 08 '16 at 20:38

I have run into this in the project I'm working on right now, and I implemented a quick extension to the ConfigParser.SafeConfigParser class in which I have overwritten the get() function. I thought some may find it useful.

import re
import ConfigParser

class ExtParser(ConfigParser.SafeConfigParser):
     #implementing extended interpolation
     def __init__(self, *args, **kwargs):
         self.cur_depth = 0 
         ConfigParser.SafeConfigParser.__init__(self, *args, **kwargs)


     def get(self, section, option, raw=False, vars=None):
         r_opt = ConfigParser.SafeConfigParser.get(self, section, option, raw=True, vars=vars)
         if raw:
             return r_opt

         ret = r_opt
         re_oldintp = r'%\((\w*)\)s'
         re_newintp = r'\$\{(\w*):(\w*)\}'

         m_new = re.findall(re_newintp, r_opt)
         if m_new:
             for f_section, f_option in m_new:
                 self.cur_depth = self.cur_depth + 1 
                 if self.cur_depth < ConfigParser.MAX_INTERPOLATION_DEPTH:
                     sub = self.get(f_section, f_option, vars=vars)
                     ret = ret.replace('${{{0}:{1}}}'.format(f_section, f_option), sub)
                 else:
                     raise ConfigParser.InterpolationDepthError, (option, section, r_opt)



         m_old = re.findall(re_oldintp, r_opt)
         if m_old:
             for l_option in m_old:
                 self.cur_depth = self.cur_depth + 1 
                 if self.cur_depth < ConfigParser.MAX_INTERPOLATION_DEPTH:
                     sub = self.get(section, l_option, vars=vars)
                     ret = ret.replace('%({0})s'.format(l_option), sub)
                 else:
                     raise ConfigParser.InterpolationDepthError, (option, section, r_opt)

         self.cur_depth = self.cur_depth - 1 
         return ret

Python ConfigParser interpolation from foreign section

4 Answers4

Edit:

Linked