0

I have a very, very large dictionary of dictionaries. Often the values are the same, and it seems there should be a way to reduce the size by having a reference to the dictionary value that is the same.

Currently I do this with a two-pass method of "Does value have synonym" followed by look up value of synonym.

But ideally it would be great to have a way to do this in a single go.

    animals = {
    'cat':{'legs':4,'eyes':2},
    'dog':{'legs':4,'eyes':2},
    'spider':{'legs':8,'eyes':6},
     }

I could have a value "mammal" that is used such that I said 'cat':mammal, but what I'd like to be able to do is 'dog':animals['cat']

Because as a reference it should take up less memory which is the goal.

I am contemplating a Class to handle this, but I can't be the first person to think that repeated values in a dictionary could be "squished" somehow, and would prefer to do it in the most pythonic way.

Brandon Wirtz
  • 43
  • 5

1 Answers1

0

I think object and inheritance are the better way for doing what you want, except maybe for the concern of memory.

For using reference instead of copying the values of each dictionary, you can use the ctypes module:

import ctypes
animals = {'cat':{'legs':4,'eyes':2},'spider':{'legs':8,'eyes':6}}
# You put the value of animals['cat'] in ['dog']
animals['dog'] = id(animals['cat'])
animals
{'dog': 47589527749808, 'spider': {'eyes': 6, 'legs': 8}, 'cat': {'eyes': 2, 'legs': 4}}
# You can access to ['dog'] with
ctypes.cast(animals['dog'], ctypes.py_object).value
{'eyes': 2, 'legs': 4}

Not sure if it is the "most pythonic way" btw. Imho class are the right way to do this.

Another way can by with using the weakref module. I don't know a lot about this one, look this post and the different answers for others hints about using reference.

Community
  • 1
  • 1
Liad
  • 340
  • 4
  • 15
  • This is helpful in at least thinking about how I might do this. I had started with a dictionary of potential values, and a dictionary of keys with a string that points to the dict of potential values. This requires to dictionary looksups, and storing the strings erodes some of the memory advantage. I am going to learn a bit about CTypes, it may be that I can do some sort of hybrid of my solution and yours. -Thanks – Brandon Wirtz Feb 08 '17 at 22:18