0

I am trying to optimize a function with nevergrad and save the resulting recommendation in a pickle. But when I assert saved_obj == loaded_obj it raises an error.

To reproduce the issue I uploaded per_recommendation.pkl : https://drive.google.com/file/d/1bqxO2JjrTP2qh23HT-qdr9Mf1Kfe4mtC/view?usp=sharing

import pickle
import nevergrad

# load obj (in the real code this is the result of some optimization)
with open('/home/franchesoni/Downloads/per_recommendation.pkl', 'rb') as f:
  r2 = pickle.load(f)
# r2 = optimizer.minimize(fn)

# save the object
with open('/home/franchesoni/Downloads/per_recommendation2.pkl', 'wb') as f:
  pickle.dump(r2, f)

# load the object
with open('/home/franchesoni/Downloads/per_recommendation2.pkl', 'rb') as f:
  recommendation = pickle.load(f)

# they are different!
assert r2 == recommendation

Is this normal or expected?

off-question: in the python docs I read pickle is unsafe, is it dangerous to open (for example) the file I uploaded? is it dangerous to reveal paths like /home/franchesoni?

Franco Marchesoni
  • 15
  • 7
  • Your data probably contains an instance of a class that doesn't compare equal to a copy of itself. Try this: `import copy`, then `r3 = copy.deepcopy(r2)`, and see if `r2` and `r3` compare equal. If not, then it should be obvious why (probably a class instance, as noted above). – Tom Karzes May 11 '20 at 19:04
  • what sort of object is this? Do you *expect* `assert r2 == recommendation`? Why? – juanpa.arrivillaga May 11 '20 at 19:12
  • @TomKarzes or just the type doesn't implement `__eq__` and does equality by identity as inherited by `object` – juanpa.arrivillaga May 11 '20 at 19:13
  • I did the test @TomKarzes proposed and indeed r2 and r3 are not equal. – Franco Marchesoni May 11 '20 at 19:52
  • @juanpa.arrivillaga I expected to restore exactly the object I saved in the first place. The object is an Instumentation object of nevergrad. I can see `__eq__` when calling dir(r) – Franco Marchesoni May 11 '20 at 19:55
  • @FrancoMarchesoni what do you mean by "exactly the object"? **Why do you expect `==`** to give you `True`? Most likely, the object doesn't implement `__eq__` and just inherits `object.__eq__` which will compare equality by identity – juanpa.arrivillaga May 11 '20 at 19:55
  • Just intuition. I've always assumed that the test proposed by @TomKarzes will pass an assertion, I see that is not the case. However, let me see if I understood your point. Is it that maybe `__eq__` was not reimplemented and that it will only pass the assertion if the names point to the same object? – Franco Marchesoni May 11 '20 at 20:04
  • @FrancoMarchesoni If you define a minimal class `Foo`, and have `f1 = Foo()` and `f2 = Foo()`, then `f1` and `f2` will not compare equal. If you add an `__eq__` method that compares two different instances for equality, then you can make `==` return `True` when appropriate. – Tom Karzes May 11 '20 at 20:10

0 Answers0