How to get a sum of a structured numpy array?

Question

I am trying to get the sum of a structured array. Do a need a "for" loop around the field names, or can I do it in one line. I do this about a million times per run, so I need to keep it quick.

>>> test = np.array([(1,2,3),(4,5,6)], dtype={'names' : ["One", "Two", "Three"], 'formats' : ["f8"]*3})
>>> test
array([(1., 2., 3.), (4., 5., 6.)],
      dtype=[('One', '<f8'), ('Two', '<f8'), ('Three', '<f8')])
>>> np.sum(test)
...
numpy.core._exceptions._UFuncNoLoopError: ufunc 'add' did not contain a loop with signature matching types (dtype([('One', '<f8'), ('Two', '<f8'), ('Three', '<f8')]), dtype([('One', '<f8'), ('Two', '<f8'), ('Three', '<f8')])) -> None

similar for np.sum(test, axis=0)

I was expecting a structured array: [(5, 7, 9), dtype=...]. At least, no errors.

You could use view(): https://stackoverflow.com/questions/54517950/element-wise-sum-of-structured-arrays-in-numpy However, note that this is not type safe. — Nick ODell, Aug 04 '23 at 22:14
As long as the number fields is relatively small (conpared to the number of records), there's little harm in iterating on them. You could also also explore using `numpy.lib.recfunctions.structured_to_unstructured` to get a 2d array. Many of the `recfunctions` iterate on fields. — hpaulj, Aug 04 '23 at 22:39

score 0 · Accepted Answer · answered Aug 04 '23 at 22:37

So, in this case, since your structure has the same primitive types, you can use a view easily enough:

>>> import numpy as np
>>> test = np.array([(1,2,3),(4,5,6)], dtype={'names' : ["One", "Two", "Three"], 'formats' : ["f8"]*3})
>>> test.view('f8').reshape(-1,3).sum(axis=0)
array([5., 7., 9.])

Now, if you couldn't easily create a view from your original structured dtype, you could use a loop over the fields, which shouldn't be terribly slow:

>>> result = np.zeros_like(test, shape=(1,))
>>> for field in test.dtype.fields:
...     result[field] = test[field].sum()
...
>>> result
array([(5., 7., 9.)],
      dtype=[('One', '<f8'), ('Two', '<f8'), ('Three', '<f8')])

That's it, thanks. I use scipy.minimize, so outside of the function, I can use the for loop. Inside, I'll use view. — engineercliff, Aug 05 '23 at 02:59

How to get a sum of a structured numpy array?

1 Answers1