1

I got a strange behaviour regarding an equal check for the weights for the vgg16 machine learning model

loading two times the model

import torch
from torch import nn
from torchvision.models import vgg16
import numpy as np
import torchvision.models as models

model = models.vgg16(weights='IMAGENET1K_V1')
torch.save(model.state_dict(), 'vgg16_model.pth')

vgg = vgg16(pretrained=True)
vgg.load_state_dict(torch.load("vgg16_model.pth", map_location='cpu'), strict=True)
params1 = np.array([param.detach().numpy() for param in vgg.parameters()], dtype=object)

vgg2 = vgg16(pretrained=True)
vgg2.load_state_dict(torch.load("vgg16_model.pth", map_location='cpu'), strict=True)
params2 = np.array([param.detach().numpy() for param in vgg2.parameters()], dtype=object)

note that I didn't replace any layer, if I do the check with np.assert_equals

np.array_equal(params1, params2)

I got False

But if I check the nested arrays iteratively the arrays are equals:

for val1, val2 in zip(params1, params2):
    print(np.array_equal(val1, val2))

What am I missing? Is it due to the way of how I create the array at the start, as dtype=object?

python version 3.9.13
numpy version 1.21.5
desertnaut
  • 57,590
  • 26
  • 140
  • 166
Nikaido
  • 4,443
  • 5
  • 30
  • 47

2 Answers2

2

Indeed, the object type seems to be the problem – note that you can even simplify your example as follows (tested with Numpy 1.24.3 on Python 3.10.10):

vgg = vgg16(pretrained=True)
vgg.load_state_dict(torch.load("vgg16_model.pth", map_location='cpu'), strict=True)
params_list = [param.detach().numpy() for param in vgg.parameters()]
params1 = np.array(params_list, dtype=object)
params2 = np.array(params_list, dtype=object)
print(np.array_equal(params1, params2))
# >>> False

That is, even if the elements inside your params1 and params2 object arrays are identical float arrays (not only equal/equivalent ones), the comparison will return False.

The comparison of Numpy arrays of type object to me seems rather unintuitive and is not very well documented. The only documentation hinting at this behavior that I could find is in some old Numpy release notes:

Object array equality comparisons

In the future object array comparisons both == and np.equal will not make use of identity checks anymore. For example:

>>> a = np.array([np.array([1, 2, 3]), 1])
>>> b = np.array([np.array([1, 2, 3]), 1])
>>> a == b

will consistently return False (and in the future an error) even if the array in a and b was the same object.

While your observed behavior is described there, the decision as to why this comparison returns False is not really motivated.

Bottom line: If you want to have element-wise comparisons of Numpy arrays, maybe don't use the object type. For further reading, maybe also have a look at this related question.

simon
  • 1,503
  • 8
  • 16
0

In 1.24

In [83]: >>> a = np.array([np.array([1, 2, 3]), 1],object)
    ...: >>> b = np.array([np.array([1, 2, 3]), 1],object)

In [84]: a==b
<ipython-input-84-a6f7ccb4d5ba>:1: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.
  a==b
Out[84]: False
hpaulj
  • 221,503
  • 14
  • 230
  • 353