'garray' object has no attribute 'size' using nolearn-DBN classifier

Question

I am working on openface. Openface has unknown classification python code.

I am testing lfw-classification-unknown.py's train part. It has training using

nolearn-DBN classifier

I installed nolearn version 0.5. DBN classifier has a function call to /usr/local/lib/python2.7/dist-packages/gnumpy.py and I have error as

File "/usr/local/lib/python2.7/dist-packages/gnumpy.py", line 738, in as_numpy_array
    if self.size==0: return numpy.zeros(self.shape, dtype)
AttributeError: 'garray' object has no attribute 'size'

How can I fix the error?

The whole errors are

Traceback (most recent call last):
  File "/usr/lib/python2.7/pdb.py", line 1314, in main
    pdb._runscript(mainpyfile)
  File "/usr/lib/python2.7/pdb.py", line 1233, in _runscript
    self.run(statement)
  File "/usr/lib/python2.7/bdb.py", line 400, in run
    exec cmd in globals, locals
  File "<string>", line 1, in <module>
  File "evaluation/lfw-classification-unknown.py", line 519, in <module>
    train(args)
  File "evaluation/lfw-classification-unknown.py", line 130, in train
    clf.fit(embeddings, labelsNum)
  File "/usr/local/lib/python2.7/dist-packages/nolearn/dbn.py", line 409, in fit
    self.use_dropout,
  File "/usr/local/lib/python2.7/dist-packages/gdbn/dbn.py", line 202, in fineTune
    err, outMB = step(inpMB, targMB, self.learnRates, self.momentum, self.L2Costs, useDropout)
  File "/usr/local/lib/python2.7/dist-packages/gdbn/dbn.py", line 296, in stepNesterov
    targetBatch = targetBatch if isinstance(targetBatch, gnp.garray) else gnp.garray(targetBatch)
  File "/usr/local/lib/python2.7/dist-packages/gnumpy.py", line 735, in __new__
    def __new__(cls, *args, **kwarg): return object.__new__(cls)
  File "/usr/lib/python2.7/bdb.py", line 53, in trace_dispatch
    return self.dispatch_return(frame, arg)
  File "/usr/lib/python2.7/bdb.py", line 88, in dispatch_return
    self.user_return(frame, arg)
  File "/usr/lib/python2.7/pdb.py", line 190, in user_return
    self.interaction(frame, None)
  File "/usr/lib/python2.7/pdb.py", line 209, in interaction
    self.print_stack_entry(self.stack[self.curindex])
  File "/usr/lib/python2.7/pdb.py", line 900, in print_stack_entry
    prompt_prefix)
  File "/usr/lib/python2.7/bdb.py", line 381, in format_stack_entry
    s = s + repr.repr(rv)
  File "/usr/lib/python2.7/repr.py", line 24, in repr
    return self.repr1(x, self.maxlevel)
  File "/usr/lib/python2.7/repr.py", line 34, in repr1
    s = __builtin__.repr(x)
  File "/usr/local/lib/python2.7/dist-packages/gnumpy.py", line 1133, in __repr__
    def __repr__(self): return self.as_numpy_array().__repr__().replace('array(', 'garray(').replace('\n', '\n ').replace(', dtype=float32', '').replace(', dtype=float64', '') # 64 happens for empty arrays
  File "/usr/local/lib/python2.7/dist-packages/gnumpy.py", line 738, in as_numpy_array
    if self.size==0: return numpy.zeros(self.shape, dtype)
AttributeError: 'garray' object has no attribute 'size'
> Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /usr/local/lib/python2.7/dist-packages/gnumpy.py(738)as_numpy_array()
-> if self.size==0: return numpy.zeros(self.shape, dtype)

EDIT: If not in Debug mode, the error is as follow.

Traceback (most recent call last):
  File "evaluation/lfw-classification-unknown.py", line 519, in <module>
    train(args)
  File "evaluation/lfw-classification-unknown.py", line 130, in train
    clf.fit(embeddings, labelsNum)
  File "/usr/local/lib/python2.7/dist-packages/nolearn/dbn.py", line 407, in fit
    self.use_dropout,
  File "/usr/local/lib/python2.7/dist-packages/gdbn/dbn.py", line 202, in fineTune
    err, outMB = step(inpMB, targMB, self.learnRates, self.momentum, self.L2Costs, useDropout)
  File "/usr/local/lib/python2.7/dist-packages/gdbn/dbn.py", line 303, in stepNesterov
    errSignals, outputActs, error = self.fpropBprop(inputBatch, targetBatch, useDropout)
  File "/usr/local/lib/python2.7/dist-packages/gdbn/dbn.py", line 262, in fpropBprop
    outputErrSignal = -self.outputActFunct.dErrordNetInput(targetBatch, self.state[-1], outputActs)
  File "/usr/local/lib/python2.7/dist-packages/gdbn/activationFunctions.py", line 138, in dErrordNetInput
    return acts - targets
  File "/usr/local/lib/python2.7/dist-packages/gnumpy.py", line 965, in __sub__
    else: return self + -as_garray(other) # if i need to broadcast, making use of the row add and col add methods is probably faster
  File "/usr/local/lib/python2.7/dist-packages/gnumpy.py", line 926, in __add__
    def __add__(self, other): return _check_number_types(self._broadcastable_op(as_garray_or_scalar(other), 'add'))
  File "/usr/local/lib/python2.7/dist-packages/gnumpy.py", line 614, in _broadcastable_op
    if reduce(operator.or_, ( other.shape[i] not in (1, self.shape[i]) for i in range(self.ndim)), False): raise ValueError('shape mismatch: objects cannot be broadcast to a single shape')
ValueError: shape mismatch: objects cannot be broadcast to a single shape

There's not enough information in your question to give a good answer. At the very least, you need to post the complete error message, including the whole stack trace (instead of just the last part). Really, you need to add a [Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve) (eg some code that will raise the same error) to your question, or else I don't think anyone is going to be able to figure out your problem. — tel, Jan 12 '19 at 10:06

tel · Answer 1 · 2019-01-12T11:27:57.067

~~You haven't posted any actual code, so it's literally impossible to know exactly what your problem is.~~

Edit

From your stack trace, it's clear that the problem you're having is due to pdb trying to grab and print the value of a garray immediately after garray.__new__ is called. This is before __init__ is called, so the .size attribute doesn't exist yet.

It doesn't seem like the redefinition of __new__ is even necessary in the gnumpy.garray code, so you might be able to fix your problem by just opening up /usr/local/lib/python2.7/dist-packages/gnumpy.py and then commenting out line 735 (ie the line with the definition of __new__).

Even simpler, what happens when you just run the script without using pdb (eg python lfw-classification-unknown.py)? It seems like this specific error would vanish. On the other hand, you were presumably using the debugger for an actual reason in the first place. Were you originally getting a different error? In that case, this is an XY problem, and you should probably just post a new question asking about the original error.

Unfortunately, gnumpy.garray requires an actual NVidia GPU to run, which I don't have, so I can't test these solutions myself directly.

The general problem

That being said, it seems that somehow a garray object is being created without it's .size attribute being set. This could be due a bug in your own code, or in any of 4 different packages (openface plus 3 dependencies). Here's a very general overview of how that might be happening.

openface has a dependency on nolearn, which has a dependency on gbdn, which has a dependency on gnumpy. In detail:

The openface code you link to has a dependency on the nolearn.dbn.DBN class.
nolearn.dbn.DBN has a dependency on the gdbn.dbn.buldDBN function, and through it the gdbn.dbn.DBN class.
Both gdbn.dbn.buldDBN and gdbn.dbn.DBN have code that creates arrays of class gnumpy.garray

So, presumably, something is getting screwed up with the creation of one of the garrays's in buildDBN or DBN. The .size attribute only gets set when the gnumpy.garray._set_shape_info method is called. A cursory look over the implementation of garray doesn't reveal any obvious way in which _set_shape_info can fail to be called during initialization. However, one thing that does jump out is that there's at least a dozen different code paths by which initialization of a garray may occur. It wouldn't exactly be surprising if there's an edge case that slips past without calling _set_shape_info.

I am using this code `https://github.com/cmusatyalab/openface/blob/master/evaluation/lfw-classification-unknown.py`. Not my own code. When you do training with this command, mentioned in this link `https://github.com/cmusatyalab/openface/pull/171`, I have such error training with `clfChoice == 'DBN':` — batuman, Jan 12 '19 at 09:23
You need to add these details to your original question. If you're using their `lfw-classification-unknown.py` script, include the exact command line that you're running the script with. — tel, Jan 12 '19 at 09:32
I posted an edit addressing the info in your complete stack trace. It's clear that the `pdb` debugger itself is causing your specific issue. What happens when you run the script without the debugger? — tel, Jan 12 '19 at 11:28
If I run not in Debug mode, the error is different. The error is shape mismatch. I have updated in Edit. — batuman, Jan 12 '19 at 12:39
You do have an [XY problem](https://mywiki.wooledge.org/XyProblem). Your actual problem has nothing to do with the question you asked. Instead of making this question an even more confusing mess, you should just ask a new question. Otherwise, it'll be too confusing for anyone who has a similar problem to find/read through. — tel, Jan 12 '19 at 12:54

'garray' object has no attribute 'size' using nolearn-DBN classifier

1 Answers1

Edit

The general problem