70

I'm new to Python, and am sort of surprised I cannot do this.

dictionary = {
    'a' : '123',
    'b' : dictionary['a'] + '456'
}

I'm wondering what the Pythonic way to correctly do this in my script, because I feel like I'm not the only one that has tried to do this.

EDIT: Enough people were wondering what I'm doing with this, so here are more details for my use cases. Lets say I want to keep dictionary objects to hold file system paths. The paths are relative to other values in the dictionary. For example, this is what one of my dictionaries may look like.

dictionary = {
    'user': 'sholsapp',
    'home': '/home/' + dictionary['user']
}

It is important that at any point in time I may change dictionary['user'] and have all of the dictionaries values reflect the change. Again, this is an example of what I'm using it for, so I hope that it conveys my goal.

From my own research I think I will need to implement a class to do this.

sholsapp
  • 15,542
  • 10
  • 50
  • 67
  • 1
    Could I make the expression lazy to fix this? – sholsapp Sep 17 '10 at 19:33
  • 1
    Do you want the value of `dictionary['b']` to be static or dynamic? What would `dictionary` look like after `dictionary['a'] = '999'`? – Will McCutchen Sep 17 '10 at 19:40
  • @Will: I was just about to edit my post to answer this question. I want it to be static, and realize that g.d.d.c's answer would by dynamic. :/ – sholsapp Sep 17 '10 at 19:41
  • Related: [Uses of self-referencing lists](http://stackoverflow.com/questions/3728667/uses-of-self-referencing-lists) – ire_and_curses Sep 17 '10 at 19:55
  • I can't fathom why you would want to do this, instead of just using local variables for your values that are related to other values. But, since you ask the Pythonic way to do this, it's certainly Daniel DiPaolo's approach. – Jeffrey Harris Sep 17 '10 at 20:20
  • I'm with Jeffrey: There's got to be a better way to do whatever it is you're trying to accomplish. If you provided more details, we might be able to help you out. – Will McCutchen Sep 17 '10 at 20:48
  • 1
    Are there languages where you can use a value from what you're defining while defining it? (so you can drive while you drive) – Nick T Sep 17 '10 at 21:07
  • 1
    Where's "g.d.d.c's answer" that everyone is referring to? – martineau Sep 17 '10 at 22:25
  • @Will McCutchen, I updated the post with a use case. @Nick, I can do this in F# or Scheme. @martineau, He must have deleted the post because it didn't compile on Python 2.4 or Python 2.6 (the latter version being the version he *claimed* it worked on). His solution was my preferred solution, though... – sholsapp Sep 17 '10 at 23:12
  • 1
    @martineau: It was withdrawn/deleted due to a misunderstanding. –  Sep 18 '10 at 02:11
  • Note that if you're actually going to be building paths as in your example, you should do it with `os.path.join`, especially if there's any chance your code will be run under Windows. – intuited Sep 19 '10 at 02:18
  • @intuited. Good point. The differing os path separator characters could be handled by adding an entry in the dictionary set to the value of `os.sep` and referring to it. For example, using the accepted answer below, you could do something like `'home' : '%{sep}shome%{sep}s%{user}s',`. – martineau Sep 19 '10 at 08:18
  • That’s still unix-specific. Use os.path.expanduser('~') to get the user home dir, and derive all paths from it (Windows does not have a single filesystem root). – merwok May 17 '12 at 04:05

11 Answers11

60

No fear of creating new classes - You can take advantage of Python's string formating capabilities and simply do:

class MyDict(dict):
   def __getitem__(self, item):
       return dict.__getitem__(self, item) % self

dictionary = MyDict({

    'user' : 'gnucom',
    'home' : '/home/%(user)s',
    'bin' : '%(home)s/bin' 
})


print dictionary["home"]
print dictionary["bin"]
aaronasterling
  • 68,820
  • 20
  • 127
  • 125
jsbueno
  • 99,910
  • 10
  • 151
  • 209
  • Are those "% dictionary" in the print statements an accidental oversight? They shouldn't be needed. –  Sep 18 '10 at 02:28
  • 2
    In the strictest since this *is* creating a new class, although it's a derived one...and nevertheless extremely clever IMHO and almost transparent in most respects (another good thing). – martineau Sep 19 '10 at 08:02
  • 1
    @Martineau: I know this is reating a new class, and this is what I meant - it was a response to Tony Veijalainen's answer (bellow) which was avoiding creating a new class. – jsbueno Sep 20 '10 at 00:49
  • 1
    This class could be made into a kind of mini python-based-spreadsheet and be able to do much more than just text substitution and string concatenation by changing the return value of `__getitem__()` to something like `eval(dict.__getitem__(self, item) % self, globals())`. Of course for safety you'd probably want to restrict what's in the dictionary argument passed and not really use globals() which has all the builtins in it. – martineau Sep 20 '10 at 13:26
  • Looking back at this now - this thing really is a minimal, super functional, reactive programming pattern. It sure could be expanded in something very useful. – jsbueno Feb 16 '18 at 14:29
  • 2
    jsbueno: It's nice you like your answer so much. `;¬)` To underscore my earlier comment about it potentially being very spreadsheet-like, here's a [recipe](http://code.activestate.com/recipes/355045-spreadsheet/) with a more complete implementation of that idea (by one of Python's core developers). – martineau Feb 16 '18 at 16:47
23

Nearest I came up without doing object:

dictionary = {
    'user' : 'gnucom',
    'home' : lambda:'/home/'+dictionary['user'] 
}

print dictionary['home']()
dictionary['user']='tony'
print dictionary['home']()
Tony Veijalainen
  • 5,447
  • 23
  • 31
  • Won't work unless `dictionary['home']()` is always used instead of the usual `dictionary['home']` method—which mean, of course, it likely won't work in code that hasn't (or can't be) changed to do that... – martineau Dec 16 '18 at 03:37
8
>>> dictionary = {
... 'a':'123'
... }
>>> dictionary['b'] = dictionary['a'] + '456'
>>> dictionary
{'a': '123', 'b': '123456'}

It works fine but when you're trying to use dictionary it hasn't been defined yet (because it has to evaluate that literal dictionary first).

But be careful because this assigns to the key of 'b' the value referenced by the key of 'a' at the time of assignment and is not going to do the lookup every time. If that is what you are looking for, it's possible but with more work.

Daniel DiPaolo
  • 55,313
  • 14
  • 116
  • 115
  • 3
    You could use `dictionary.update({'b': dictionary['a'] + '456'})` and pass in as big of a dictionary as you want, but it sounds like g.d.d.c has the approach you want, which is truly self-referential and dynamic lookup and not just "at-assignment-time" lookup. – Daniel DiPaolo Sep 17 '10 at 19:59
  • Thanks for the info. That may be what I end up doing, because we have Python 2.4 installed by default. – sholsapp Sep 17 '10 at 20:01
  • 4
    @gnucom: python 2.4 is quite old, the current version is 2.7, and all the real effort is focused on 3. You should seriously try to upgrade. – Daenyth Sep 17 '10 at 23:37
7

What you're describing in your edit is how an INI config file works. Python does have a built in library called ConfigParser which should work for what you're describing.

Greg
  • 45,306
  • 89
  • 231
  • 297
6

This is an interesting problem. It seems like Greg has a good solution. But that's no fun ;)

jsbueno as a very elegant solution but that only applies to strings (as you requested).

The trick to a 'general' self referential dictionary is to use a surrogate object. It takes a few (understatement) lines of code to pull off, but the usage is about what you want:

S = SurrogateDict(AdditionSurrogateDictEntry)
d = S.resolve({'user': 'gnucom',
               'home': '/home/' + S['user'],
               'config': [S['home'] + '/.emacs', S['home'] + '/.bashrc']})

The code to make that happen is not nearly so short. It lives in three classes:

import abc

class SurrogateDictEntry(object):
    __metaclass__ = abc.ABCMeta
    def __init__(self, key):
        """record the key on the real dictionary that this will resolve to a 
           value for
        """
        self.key = key

    def resolve(self, d):
        """ return the actual value"""
        if hasattr(self, 'op'):
            # any operation done on self will store it's name in self.op. 
            # if this is set, resolve it by calling the appropriate method 
            # now that we can get self.value out of d
            self.value = d[self.key]
            return getattr(self, self.op + 'resolve__')()
        else:
            return d[self.key]

    @staticmethod
    def make_op(opname):
        """A convience class. This will be the form of all op hooks for subclasses
           The actual logic for the op is in __op__resolve__ (e.g. __add__resolve__)
        """
        def op(self, other):
            self.stored_value = other
            self.op = opname
            return self
        op.__name__ = opname
        return op

Next, comes the concrete class. simple enough.

class AdditionSurrogateDictEntry(SurrogateDictEntry):

    __add__ = SurrogateDictEntry.make_op('__add__')
    __radd__ = SurrogateDictEntry.make_op('__radd__')

    def __add__resolve__(self):
        return self.value + self.stored_value 

    def __radd__resolve__(self):
        return self.stored_value + self.value

Here's the final class

class SurrogateDict(object):
    def __init__(self, EntryClass):
        self.EntryClass = EntryClass

    def __getitem__(self, key):
        """record the key and return""" 
        return self.EntryClass(key)

    @staticmethod
    def resolve(d):
        """I eat generators resolve self references"""
        stack = [d]
        while stack:
            cur = stack.pop()
            # This just tries to set it to an appropriate iterable
            it = xrange(len(cur)) if not hasattr(cur, 'keys') else cur.keys()
            for key in it:
                # sorry for being a duche. Just register your class with
                # SurrogateDictEntry and you can pass whatever.
                while isinstance(cur[key], SurrogateDictEntry):
                    cur[key] = cur[key].resolve(d)
                # I'm just going to check for iter but you can add other
                # checks here for items that we should loop over. 
                if hasattr(cur[key], '__iter__'):
                    stack.append(cur[key])
        return d

In response to gnucoms's question about why I named the classes the way that I did.

The word surrogate is generally associated with standing in for something else so it seemed appropriate because that's what the SurrogateDict class does: an instance replaces the 'self' references in a dictionary literal. That being said, (other than just being straight up stupid sometimes) naming is probably one of the hardest things for me about coding. If you (or anyone else) can suggest a better name, I'm all ears.

I'll provide a brief explanation. Throughout S refers to an instance of SurrogateDict and d is the real dictionary.

  1. A reference S[key] triggers S.__getitem__ and SurrogateDictEntry(key) to be placed in the d.

  2. When S[key] = SurrogateDictEntry(key) is constructed, it stores key. This will be the key into d for the value that this entry of SurrogateDictEntry is acting as a surrogate for.

  3. After S[key] is returned, it is either entered into the d, or has some operation(s) performed on it. If an operation is performed on it, it triggers the relative __op__ method which simple stores the value that the operation is performed on and the name of the operation and then returns itself. We can't actually resolve the operation because d hasn't been constructed yet.

  4. After d is constructed, it is passed to S.resolve. This method loops through d finding any instances of SurrogateDictEntry and replacing them with the result of calling the resolve method on the instance.

  5. The SurrogateDictEntry.resolve method receives the now constructed d as an argument and can use the value of key that it stored at construction time to get the value that it is acting as a surrogate for. If an operation was performed on it after creation, the op attribute will have been set with the name of the operation that was performed. If the class has a __op__ method, then it has a __op__resolve__ method with the actual logic that would normally be in the __op__ method. So now we have the logic (self.op__resolve) and all necessary values (self.value, self.stored_value) to finally get the real value of d[key]. So we return that which step 4 places in the dictionary.

  6. finally the SurrogateDict.resolve method returns d with all references resolved.

That'a a rough sketch. If you have any more questions, feel free to ask.

Community
  • 1
  • 1
aaronasterling
  • 68,820
  • 20
  • 127
  • 125
  • Can you possibly discuss why you named your classes the way you did? I'm trying to develop a thorough understanding of the class and, well, I don't understand every part of your code. :P – sholsapp Sep 19 '10 at 01:03
  • @gnucom, updated the bottom of my post. HTH. feel free to ask any questions if something is tripping you up. It's a little fragile right now. In particular, `L[foo] + 4 + 4` might fail. `L[3] + L[3] + 1` works though. I'm not sure about `1 + L[3] + l` but I think that that would fail. The two (alleged) failures could be corrected by having `SurrogateDictEntry` implement an operation stack instead of just storing one operation/value pair. – aaronasterling Sep 19 '10 at 02:13
  • @gnucom Also don't get too infatuated with this solution. The two that I linked to are __much__ better for your use-case. I mainly did it just to do it. – aaronasterling Sep 19 '10 at 02:35
4

If you, just like me wandering how to make @jsbueno snippet work with {} style substitutions, below is the example code (which is probably not much efficient though):

import string

class MyDict(dict):
    def __init__(self, *args, **kw):
        super(MyDict,self).__init__(*args, **kw)
        self.itemlist = super(MyDict,self).keys()
        self.fmt = string.Formatter() 

    def __getitem__(self, item):
        return self.fmt.vformat(dict.__getitem__(self, item), {}, self)


xs = MyDict({
    'user' : 'gnucom',
    'home' : '/home/{user}',
    'bin' : '{home}/bin'
})


>>> xs["home"]
'/home/gnucom'
>>> xs["bin"]
'/home/gnucom/bin'

I tried to make it work with the simple replacement of % self with .format(**self) but it turns out it wouldn't work for nested expressions (like 'bin' in above listing, which references 'home', which has it's own reference to 'user') because of the evaluation order (** expansion is done before actual format call and it's not delayed like in original % version).

Community
  • 1
  • 1
om-nom-nom
  • 62,329
  • 13
  • 183
  • 228
3

Write a class, maybe something with properties:

class PathInfo(object):
    def __init__(self, user):
        self.user = user

    @property
    def home(self):
        return '/home/' + self.user

p = PathInfo('thc')
print p.home # /home/thc 
Jochen Ritzel
  • 104,512
  • 31
  • 200
  • 194
2

As sort of an extended version of @Tony's answer, you could build a dictionary subclass that calls its values if they are callables:

class CallingDict(dict):
    """Returns the result rather than the value of referenced callables.

    >>> cd = CallingDict({1: "One", 2: "Two", 'fsh': "Fish",
    ...                   "rhyme": lambda d: ' '.join((d[1], d['fsh'],
    ...                                                d[2], d['fsh']))})
    >>> cd["rhyme"]
    'One Fish Two Fish'
    >>> cd[1] = 'Red'
    >>> cd[2] = 'Blue'
    >>> cd["rhyme"]
    'Red Fish Blue Fish'
    """
    def __getitem__(self, item):
        it = super(CallingDict, self).__getitem__(item)
        if callable(it):
            return it(self)
        else:
            return it

Of course this would only be usable if you're not actually going to store callables as values. If you need to be able to do that, you could wrap the lambda declaration in a function that adds some attribute to the resulting lambda, and check for it in CallingDict.__getitem__, but at that point it's getting complex, and long-winded, enough that it might just be easier to use a class for your data in the first place.

Community
  • 1
  • 1
intuited
  • 23,174
  • 7
  • 66
  • 88
1

This is very easy in a lazily evaluated language (haskell).

Since Python is strictly evaluated, we can do a little trick to turn things lazy:

Y = lambda f: (lambda x: x(x))(lambda y: f(lambda *args: y(y)(*args)))

d1 = lambda self: lambda: {
  'a': lambda: 3,
  'b': lambda: self()['a']()
}

# fix the d1, and evaluate it
d2 = Y(d1)()

# to get a
d2['a']() # 3

# to get b
d2['b']() # 3

Syntax wise this is not very nice. That's because of us needing to explicitly construct lazy expressions with lambda: ... and explicitly evaluate lazy expression with ...(). It's the opposite problem in lazy languages needing strictness annotations, here in Python we end up needing lazy annotations.

I think with some more meta-programmming and some more tricks, the above could be made more easy to use.

Note that this is basically how let-rec works in some functional languages.

CMCDragonkai
  • 6,222
  • 12
  • 56
  • 98
1

The jsbueno answer in Python 3 :

class MyDict(dict):
    def __getitem__(self, item):
        return dict.__getitem__(self, item).format(self)

dictionary = MyDict({
    'user' : 'gnucom',
    'home' : '/home/{0[user]}',
    'bin' : '{0[home]}/bin' 
})

print(dictionary["home"])
print(dictionary["bin"])

Here we use the python 3 string formatting with curly braces {} and the .format() method.

Documentation : https://docs.python.org/3/library/string.html

Rivers
  • 1,783
  • 1
  • 8
  • 27
0

Class statements can be used to define dictionaries with string keys in multiple steps like this:

def data(name,bases,values): return values
class dictionary(metaclass=data):
  a='123'
  b=a+'456'

The “metaclass” just uses the locals of the class statement as the value of the class definition (which is assigned here to dictionary). This approach avoids relying on dict.__getitem__ in a way that will be inconsistent with other access via dict.get, dict.popitem, etc.

Like several other answers here, though, it does not support the “spreadsheet” use case of redefining values implicitly when other values change. (That sort of object isn’t a dictionary of values at all, but rather of nullary functions that can be evaluated when desired.)

Davis Herring
  • 36,443
  • 4
  • 48
  • 76