0

I'm trying klepto as a replacement for shelve to persist class instances but am seeing a lengthy traceback when attempting to do so.

My platform is: python 3.7.3 klepto 0.1.6

The example @ Pickle versus shelve storing large dictionaries in Python ... works fine on the above platform.

from   dataclasses import dataclass
@dataclass (frozen=True)
class Target:
    name : str
@dataclass (frozen=True)
class Image:
    target :Target

image_1 = Image (target=Target(name='Crab Nebula'))
image_2 = Image (target=Target(name='Orion Nebula'))
##keys = image_1.name, image_2.name
keys = image_1.target.name, image_2.target.name
values = image_1, image_2
d = dict ( zip (keys,values))
import klepto
a = klepto.archives.dir_archive (dict=d)
a.dump()

a.dump() initiates a lengthy traceback beginning with ...

File "C:\Users\jgv\Documents\Projects\AstroPix_ODB (klepto)\stackoverflow.py", line 20, in <module>
   a.dump()
 File "C:\Users\jgv\AppData\Local\Programs\Python\Python37\lib\site-packages\klepto\_archives.py", line 165, in dump
   self.archive.update(self)

and concluding with ...

 File "C:\Users\jgv\AppData\Local\Programs\Python\Python37\lib\site-packages\dill\_dill.py", line 1148, in save_dictproxy
   raise ReferenceError("%s does not reference a class __dict__" % obj)
ReferenceError: {} does not reference a class __dict__
jim vickroy
  • 142
  • 2
  • 9
  • FWIW removing the @dataclass decorators and defining the classes in un-decorated fashion works. – jim vickroy May 05 '19 at 01:18
  • Hi I'm the `klepto` author. The reason you are seeing that error is that `dill` doesn't know how to serialize a `dataclass` . So, if you use an `serialized=False` archive, or some other `protocol`, such as `json` or `fast=True`, you can successfully `dump` the object. There are a lot of format variants available in `klepto`. Looks like, regardless, there are still some failures on `load`. Can you report this to the `klepto` GitHub page? – Mike McKerns May 05 '19 at 16:05
  • Thanks for the quick response, Mike. An issue "dill doesn't know how to serialize @dataclass instances" has been submitted on the klepto GitHub page. – jim vickroy May 06 '19 at 00:59
  • Ah, yes, that is a solution. I think there's a second issue... `klepto` appears to botching the non-serialized attempt to store the object as well. Could you also report failure to dump/load the object when using `serialized=False`? BTW: using whitespace in a key's name is not a good idea unless you plan on encoding the key. – Mike McKerns May 06 '19 at 12:21
  • I see, you reported the `dill` issue to `klepto`. No worries. I adjusted issues accordingly. – Mike McKerns May 06 '19 at 13:04
  • Could you clarify why keys with (embedded) white-space are a bad idea? The actual keys are "pre-treated" to strip leading/trailing white-space and reduce all embedded multi-space to single-space. Does klepto support non-string keys (i.e., hash-able class instances)? – jim vickroy May 06 '19 at 17:16
  • Jim, it depending on the key encoding. Some keys are encoded to hashes, some to compact sequences with no whitespace, some serialized objects, and others are treated as mildly-mangled strings. So, yes, the `dir_archive` with a particular encoding could set the name of the file to something with a embedded whitespace, which wouldn't be good. Other encodings wouldn't care. – Mike McKerns May 06 '19 at 18:17

0 Answers0