Total size of array must be unchanged

Question

I am using a Python module called emcee to sample a distribution. I need to pass a (37,100) (which I have named Ntrig and Nsamp, respectively) array called events to the below function.

def mp(SNR2, *events):
    events = np.asarray(events).reshape((Ntrig,Nsamp))
    bessel = special.iv(0,np.sqrt(x*SNR2(event)))
    exp = np.exp(-0.5*(x+SNR2(event)))
    I = integrate.quad(lambda x: exp*bessel,0,SNRth**2)[0]
    return np.asarray([np.array[I for event in events[i]] for i in range(len(events))]).reshape(events.shape)

I keep getting the error:

ValueError: total size of new array must be unchanged

As I understand, *events will break up the events array into 37*100 separate arguments. Shouldn't the next line where I reshape the array just put it back into a 37 by 100 array?

P.S. before you ask why I even bother breaking up events into separate arguments--the module needs this to work, it can't take an array.

Full Traceback error:

ValueError                                Traceback (most recent call last)
<ipython-input-17-c8e815326a69> in <module>()
----> 1 mp(SNR2,events)

<ipython-input-16-9f73f234c628> in mp(SNR2, *events)
      5 def mp(SNR2, *events):
      6     events = np.asarray(events).reshape((Ntrig,Nsamp))
----> 7     return np.asarray([np.array([integrate.quad(lambda x: np.exp(-0.5*(x+SNR2(event)))*special.iv(0,np.sqrt(x*SNR2(event))),0,SNRth**2)[0] for event in events[i]]) for i in range(len(events))]).reshape(events.shape)
      8 #    return integrate.quad(lambda x: 0.5*np.exp(-0.5*(x+SNR2(event)))*special.iv(0,np.sqrt(x*SNR2(event))),0,SNRth**2)[0]
      9 def pp(SNR2, *events):

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/integrate/quadpack.pyc in quad(func, a, b, args, full_output, epsabs, epsrel, limit, points, weight, wvar, wopts, maxp1, limlst)
    279         args = (args,)
    280     if (weight is None):
--> 281         retval = _quad(func,a,b,args,full_output,epsabs,epsrel,limit,points)
    282     else:
    283         retval = _quad_weight(func,a,b,args,full_output,epsabs,epsrel,limlst,limit,maxp1,weight,wvar,wopts)

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/integrate/quadpack.pyc in _quad(func, a, b, args, full_output, epsabs, epsrel, limit, points)
    343     if points is None:
    344         if infbounds == 0:
--> 345             return _quadpack._qagse(func,a,b,args,full_output,epsabs,epsrel,limit)
    346         else:
    347             return _quadpack._qagie(func,bound,infbounds,args,full_output,epsabs,epsrel,limit)

<ipython-input-16-9f73f234c628> in <lambda>(x)
      5 def mp(SNR2, *events):
      6     events = np.asarray(events).reshape((Ntrig,Nsamp))
----> 7     return np.asarray([np.array([integrate.quad(lambda x: np.exp(-0.5*(x+SNR2(event)))*special.iv(0,np.sqrt(x*SNR2(event))),0,SNRth**2)[0] for event in events[i]]) for i in range(len(events))]).reshape(events.shape)
      8 #    return integrate.quad(lambda x: 0.5*np.exp(-0.5*(x+SNR2(event)))*special.iv(0,np.sqrt(x*SNR2(event))),0,SNRth**2)[0]
      9 def pp(SNR2, *events):

<ipython-input-16-9f73f234c628> in SNR2(*events)
      1 def SNR2(*events):
----> 2     events = np.asarray(events).reshape((Ntrig,Nsamp))
      3     C = 5*np.pi**(-1.33333)*events**(1.66667)/(96*d**2)
      4     return C*integrate.quad(lambda f: f**(-2.3333)/S(f), 20, 1500, limit=1000)[0]
      5 def mp(SNR2, *events):

ValueError: total size of new array must be unchanged

unutbu · Accepted Answer · 2014-07-23T20:07:08.710

3

As I understand, events will break up the events array into 37*100 separate arguments.

This is not true. If you call mp using

mp(SNR2, events)

then inside mp, events will be a 1-element tuple, (arr,), where arr is the (37, 100)-shaped array.

If you call mp using

mp(SNR2, *events)

then inside mp, events will be a 37-element tuple, where the 37 elements are the 37 rows of the (37, 100)-shaped array.

If you call mp using

mp(SNR2, *events.flat)

then inside mp, events will be a tuple of 37*100 elements.

Notice that the last stanza of the traceback says:

<ipython-input-16-9f73f234c628> in SNR2(*events)
      1 def SNR2(*events):
----> 2     events = np.asarray(events).reshape((Ntrig,Nsamp))
      3     C = 5*np.pi**(-1.33333)*events**(1.66667)/(96*d**2)
      4     return C*integrate.quad(lambda f: f**(-2.3333)/S(f), 20, 1500, limit=1000)[0]
      5 def mp(SNR2, *events):

ValueError: total size of new array must be unchanged

So the error is raised while Python is in the SNR2 function.

Since SNR2 was called in mp using SNR2(event), and event is a (37,100)-shaped array, the event variable in SNR2 is a 1-element tuple containing the original array. That's not what you want.

The easiest way to fix the code is to define

def SNR2(events):  
    # no longer needed
    # events = np.asarray(events).reshape((Ntrig,Nsamp))

and just pass events around as one would expect.

However, if you can not change the signature of SNR2, then you must call it with

SNR2(*event.flat)

inside the mp function.

Reference: Here is an excellent explanation of the * unpacking operator, and how the syntax is used when defining functions, and calling functions.

edited Jul 23 '14 at 20:07

answered Jul 23 '14 at 18:52

unutbu

842,883
184
1,785
1,677

Why am I getting the error `ValueError: total size of new array must be unchanged` and how do I fix it? – naomig Jul 23 '14 at 18:56
Are you just asking "how do I force the array to break up into 37*100 separate elements"? If so, `*iter(events)` will do that, or `*list(events)`, or `*(events.reshape((37*100,))`, etc. – abarnert Jul 23 '14 at 18:57
I don't know. On what line are you getting that error? Please post the full traceback error message. – unutbu Jul 23 '14 at 18:57
I'm getting the error on the second line of the code, where I reshape `events`. `events` is already a (37,100) array. When I input it into the function, I need to use *events. I then want to reshape events back into a (37,100) array. When I try to do this, however, I get the aforementioned error. – naomig Jul 23 '14 at 19:41
I should mention: when I test this simple code: `def func(*events): events = np.asarray(events).reshape((Ntrig,Nsamp)) return events` there is no problem. – naomig Jul 23 '14 at 19:44
@unutbu I editted the original post with the full traceback error. – naomig Jul 23 '14 at 19:50
That seems very odd -- maybe a contradiction. Since Python executes each statement line-by-line, if `mp` and `func` are identical up to the call to `reshape`, then one should not cause an Exception while the other does not. – unutbu Jul 23 '14 at 19:50
@unutbu what do you mean by contradiction? And yes, it's odd, it's been really eating at me haha – naomig Jul 23 '14 at 19:51
Ohh.. The exception is raised in SNR2, not mp! – unutbu Jul 23 '14 at 19:52
@unutbu but SNR2 runs perfectly! SNR2(events) outputs a (37,100) array – naomig Jul 23 '14 at 19:53

Total size of array must be unchanged

1 Answers1