2

In ZOBD (in Python 3.x) I would like to be able to store objects as keys in an BTrees.OOBTree.OOBTree(). Example of the error I get when I try (see the comment):

from BTrees.OOBTree import OOBTree as Btree

class Test:
    pass

bt=Btree()
t=Test()
bt[t]=None #TypeError: Object has default comparison

So, I read somewhere that __eq__ may need to be defined to remove that error, but although that seemed to fix the previous problem it seems to cause more problems. Example:

[EDIT: It should be noted that I've found some problems with inheriting OOBTree (and TreeSet) as I do here. Apparently, they don't save properly; so, it's not the same as inheriting Persistent, even though they inherit Persistent.]

from BTrees.OOBTree import OOBTree as Btree

class Test:
    def __eq__(self, other): #Maybe this isn't the way to define the method
        return self==other

bt=Btree()
t=Test()
bt[t]=None

t in bt #TypeError: unorderable types: Test() < Test()

What is the correct way to use objects as keys in a BTree or OOBTree? I do need to test whether the key exists, too.

For those who don't know, BTrees in ZODB are pretty much like scalable Python dictionaries (they should be workable with more key-value pairs than a regular Python dictionary) designed for persistence.

Brōtsyorfuzthrāx
  • 4,387
  • 4
  • 34
  • 56

2 Answers2

4

I think this answer can help with your problem.

Bascically, you have to reimplent three methods on your object:

  1. __eq__ (equality check)
  2. __ne__ (non equality check)
  3. __hash__ to make the object really serializable as a dictionary key
Community
  • 1
  • 1
Agate
  • 3,152
  • 1
  • 19
  • 30
  • Usually it's a good idea to just go with [total_ordering](https://docs.python.org/3/library/functools.html#functools.total_ordering) – dom0 Jan 13 '17 at 13:07
1

Although Agate's answer led me to the answers I needed, I figured I would post the full answer that helped me so others don't have to spend extra time figuring stuff out. (I'm going to speak to myself in the second person.)


First of all (I didn't really ask about it, but it's something you might be tempted to do), don't inherit OOBTree or OOTreeSet (this causes problems). Make your own classes that inherit Persistent, and put an OOBTree or an OOTreeSet inside, if you want something like an inherited OOBTree (also, define the methods needed to make it seem like a dictionary or a set if you want that).

Next of all, you need to create a Persistent ID system (for every object that you put in the OOBTree or OOTreeSet, because objects cause OOBTrees and OOTreeSets to malfunction if you don't have a unique integer that ZOBD can keep track of your objects with. You need to define the methods that Eliot mentioned, as well as some other similar ones (and these need to compare that integer ID—not the object itself); i.e. define these methods of your classes that produce objects that will be keys of an OOBTree or contained in an OOTreeSet: __eq__, __ne__, __hash__, __lt__, __le__, __gt__, and __ge__. However, in order to have a persistent ID, you're going to have to make an ID counter class or something (because it won't save plain integers as values in an OOBTree for some odd reason, unless I did it wrong), and that counter class will have to have an ID, too.

Next of all, you need to make sure that if you're making objects keys, then you better not make things like strings be keys, too, in the same OOBTree, or else you'll have mysterious issues (due to strings not having the same sort of ID system as your objects). It'll be comparing the string keys to your object keys, and cause an error, because they're not designed to compare.

Here is a working example of Python 3.x code that allows you to use objects as keys in an OOBTree, and it will allow you to iterate over persistent objects in an OOBTree (and use them as keys). It also shows you how it can save and load the objects.

Sorry it's kind of long, but it should give you a good idea of how this can work:

import transaction, ZODB, ZODB.FileStorage
from persistent import Persistent
from BTrees.OOBTree import OOBTree as OOBTree
from BTrees.OOBTree import OOTreeSet as OOTreeSet

class Btree(Persistent):
    def __init__(self, ID=None, **attr):
        #I like to use entirely uppercase variables to represent ones you aren't supposed to access outside of the class (because it doesn't have the restrictions that adding _ and __ to the beginning do, and because you don't really need all caps for constants in Python)
        Persistent.__init__(self)
        self.DS=OOBTree() #DS stands for data structure
        self.DS.update(attr)
        if ID==None:
            self.ID=-1 #To give each object a unique id. The value, -1, is replaced.
            self.ID_SET=False
        else:
            self.ID=ID #You should remember what you’re putting here, and it should be negative.
            self.ID_SET=True
    def clear(self):
        self.DS.clear()
    def __delitem__(self, key):
        del self.DS[key]
    def __getitem__(self, key):
        return self.DS[key]
    def __len__(self):
        return len(self.DS)
    def __iadd__(self, other):
        self.DS.update(other)
    def __isub__(self, other):
        for x in other:
            try:
                del self.DS[x]
            except KeyError:
                pass
    def __contains__(self, key):
        return self.DS.has_key(key)
    def __setitem__(self, key, value):
        self.DS[key]=value
    def __iter__(self):
        return iter(self.DS)
    def __eq__(self, other):
        return self.id==other.id
    def __ne__(self, other):
        return self.id!=other.id
    def __hash__(self):
        return self.id
    def __lt__(self, other):
        return self.id<other.id
    def __le__(self, other):
        return self.id<=other.id
    def __gt__(self, other):
        return self.id>other.id
    def __ge__(self, other):
        return self.id>=other.id
    @property
    def id(self):
        if self.ID_SET==False:
            print("Warning. self.id_set is False. You are accessing an id that has not been set.")
        return self.ID
    @id.setter
    def id(self, num):
        if self.ID_SET==True:
            raise ValueError("Once set, the id value may not be changed.")
        else:
            self.ID=num
            self.ID_SET=True
    def save(self, manager, commit=True):
        if self.ID_SET==False:
            self.id=manager.inc()
        manager.root.other_set.add(self)
        if commit==True:
            transaction.commit()

class Set(Persistent):
    def __init__(self, ID=None, *items):
        Persistent.__init__(self)
        self.DS=OOTreeSet()
        if ID==None:
            self.ID=-1 #To give each object a unique id. The value, -1, is replaced automatically when saved by the project for the first time (which should be done right after the object is created).
            self.ID_SET=False
        else:
            if ID>=0:
                raise ValueError("Manual values should be negative.")
            self.ID=ID #You should remember what you’re putting here, and it should be negative.
            self.ID_SET=True
        self.update(items)
    def update(self, items):
        self.DS.update(items)
    def add(self, *items):
        self.DS.update(items)
    def remove(self, *items):
        for x in items:
            self.DS.remove(x)
    def has(self, *items):
        for x in items:
            if not self.DS.has_key(x):
                return False
        return True
    def __len__(self):
        return len(self.DS)
    def __iadd__(self, other):
        self.DS.update(other)
    def __isub__(self, other):
        self.remove(*other)
    def __contains__(self, other):
        return self.DS.has_key(other)
    def __iter__(self):
        return iter(self.DS)
    def __eq__(self, other):
        return self.id==other.id
    def __ne__(self, other):
        return self.id!=other.id
    def __hash__(self):
        return self.id
    def __lt__(self, other):
        return self.id<other.id
    def __le__(self, other):
        return self.id<=other.id
    def __gt__(self, other):
        return self.id>other.id
    def __ge__(self, other):
        return self.id>=other.id
    @property
    def id(self):
        if self.ID_SET==False:
            print("Warning. self.id_set is False. You are accessing an id that has not been set.")
        return self.ID
    @id.setter
    def id(self, num):
        if self.ID_SET==True:
            raise ValueError("Once set, the id value may not be changed.")
        else:
            self.ID=num
            self.ID_SET=True
    def save(self, manager, commit=True):
        if self.ID_SET==False:
            self.id=manager.inc()
        manager.root.other_set.add(self)
        if commit==True:
            transaction.commit()

class Counter(Persistent):
    #This is for creating a persistent id count object (using a plain integer outside of a class doesn't seem to work).
    def __init__(self, value=0):
        self.value=value
        self.ID_SET=False
        self.id=value
    #The following methods are so it will fit fine in a BTree (they don't have anything to do with self.value)
    def __eq__(self, other):
        return self.id==other.id
    def __ne__(self, other):
        return self.id!=other.id
    def __hash__(self):
        return self.id
    def __lt__(self, other):
        return self.id<other.id
    def __le__(self, other):
        return self.id<=other.id
    def __gt__(self, other):
        return self.id>other.id
    def __ge__(self, other):
        return self.id>=other.id
    @property
    def id(self):
        if self.ID_SET==False:
            print("Warning. self.id_set is False. You are accessing an id that has not been set.")
        return self.ID
    @id.setter
    def id(self, num):
        if self.ID_SET==True:
            raise ValueError("Once set, the id value may not be changed.")
        else:
            self.ID=num
            self.ID_SET=True

class Manager:
    def __init__(self, filepath):
        self.filepath=filepath
        self.storage = ZODB.FileStorage.FileStorage(filepath)
        self.db = ZODB.DB(self.storage)
        self.conn = self.db.open()
        self.root = self.conn.root
        print("Database opened.\n")
        try:
            self.root.other_dict #This holds arbitrary stuff, like the Counter. String keys.
        except AttributeError:
            self.root.other_dict=OOBTree()
            self.root.other_dict["id_count"]=Counter()
        try:
            self.root.other_set #set other
        except AttributeError:
            self.root.other_set=OOTreeSet() #This holds all our Btree and Set objects (they are put here when saved to help them be persistent).
    def inc(self): #This increments our Counter and returns the new value to become the integer id of a new object.
        self.root.other_dict["id_count"].value+=1
        return self.root.other_dict["id_count"].value
    def close(self):
        self.db.pack()
        self.db.close()
        print("\nDatabase closed.")

class Btree2(Btree):
    #To prove that we can inherit our own classes we created that inherit Persistent (but inheriting OOBTree or OOTreeSet causes issues)
    def __init__(self, ID=None, **attr):
        Btree.__init__(self, ID, **attr)




m=Manager("/path/to/database/test.fs")

try:
    m.root.tree #Causes an AttributeError if this is the first time you ran the program, because it doesn't exist.
    print("OOBTree loaded.")
except AttributeError:
    print("Creating OOBTree.")
    m.root.tree=OOBTree()
    for i in range(5):
        key=Btree2()
        key.save(m, commit=False) #Saving without committing adds it to the manager's OOBTree and gives it an integer ID. This needs to be done right after creating an object (whether or not you commit).
        value=Btree2()
        value.save(m, commit=False)
        m.root.tree[key]=value #Assigning key and value (which are both objects) to the OOBTree
    transaction.commit() #Commit the transactions

try:
    m.root.set
    print("OOTreeSet loaded.")
except AttributeError:
    print("Creating OOTreeSet")
    m.root.set=OOTreeSet()
    for i in range(5):
        item=Set()
        item.save(m, commit=False)
        m.root.set.add(item)
    transaction.commit()

#Doing the same with an OOTreeSet (since objects in them suffered from the same problem as objects as keys in an OOBTree)
for x in m.root.tree:
    print("Key: "+str(x.id))
    print("Value: "+str(m.root.tree[x].id))
    if x in m.root.tree:
        print("Comparison works for "+str(x.id))

print("\nOn to OOTreeSet.\n")

for x in m.root.set:
    if x in m.root.set:
        print("Comparison works for "+str(x.id))

m.close()
Agate
  • 3,152
  • 1
  • 19
  • 30
Brōtsyorfuzthrāx
  • 4,387
  • 4
  • 34
  • 56