1

I keep getting this RuntimeWarning, and I can't figure out where it's coming from, since I don't use np.nanmean anywhere in my program. I do use np.mean, though. Does anyone know if there's some other numpy function (e.g., np.mean) that calls np.nanmean, and may be throwing this warning?

I would just go and suppress it, however, each time the warning occurs, afterward when I go to fit a polynomial to the points being processed, I get the following error:

C:\Users\M.Modeler\AppData\Local\miniconda3\envs\cloudtexture\lib\site-packages\numpy\lib\nanfunctions.py:1559: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis, out=out, keepdims=keepdims)
Intel MKL ERROR: Parameter 6 was incorrect on entry to DGELSD.
Traceback (most recent call last):
  File "D:\Work_D\RD22-02_GSM\cloudTexture\cloudTexture.py", line 345, in <module>
    thinned = ct.normalize(thinned,
  File "D:\Work_D\RD22-02_GSM\cloudTexture\ctUtil.py", line 332, in normalize
    poly = np.flip(np.polyfit(blnd[p], relrough[p], deg = polyorder))
  File "<__array_function__ internals>", line 180, in polyfit
  File "C:\Users\M.Modeler\AppData\Local\miniconda3\envs\cloudtexture\lib\site-packages\numpy\lib\polynomial.py", line 668, in polyfit
    c, resids, rank, s = lstsq(lhs, rhs, rcond)
  File "<__array_function__ internals>", line 180, in lstsq
  File "C:\Users\M.Modeler\AppData\Local\miniconda3\envs\cloudtexture\lib\site-packages\numpy\linalg\linalg.py", line 2292, in lstsq
    x, resids, rank, s = gufunc(a, b, rcond, signature=signature, extobj=extobj)
  File "C:\Users\M.Modeler\AppData\Local\miniconda3\envs\cloudtexture\lib\site-packages\numpy\linalg\linalg.py", line 100, in _raise_linalgerror_lstsq
    raise LinAlgError("SVD did not converge in Linear Least Squares")
numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares

For datasets where I don't get the mean of the empty slice warning, the polyfit is successful. So it seems the nanmean warning is telling me something useful, I just can't find what's causing it. Any ideas?

Additionally, any insight into the potential triggers of the above error could be helpful in tracking down the root cause of this?

bt3
  • 156
  • 1
  • 10
  • The error in your title doesn't appear in the error trace you share. – jared Aug 18 '23 at 15:51
  • I get the RuntimeWarning in the title in a separate process (thinning a point cloud based on a voxel grid), and then move on to fit polynomials to certain aspects of the point cloud (for detrending the sonar data). Whenever the RuntimeWarning occurs, subsequently, the error in the above trace happens. – bt3 Aug 18 '23 at 15:56
  • Please include the runtime warning in your question with the traceback. – jared Aug 18 '23 at 15:57
  • Warnings don't normally show the context, a traceback. It's possible that something liike `fit` is using `np.nanmean`. If the array, or Series, has 0 length, or is entirely filled wtih `nan`, you'll get this warning, and the result will be `np.nan`. That `nan` result could be messing up downstream results. – hpaulj Aug 18 '23 at 16:34
  • Thanks. This answer: https://stackoverflow.com/a/38431803/19588737 just helped me figure out that `nan` values are being passed to `np.polyfit`. I realized I do use `np.nanquantile` in the subroutine that throws the warning, maybe `nanquantile` is calling `nanmean`. – bt3 Aug 18 '23 at 16:41

1 Answers1

2

With some testing I found that np.nanquantile receiving an array with 0 elements is what is triggering the runtime warning:

>>> import numpy as np
>>> a = np.array([])
>>> np.nanquantile(a, .1)
C:\Users\M.Modeler\AppData\Local\miniconda3\envs\cloudtexture\lib\site-packages\numpy\lib\nanfunctions.py:1559: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis, out=out, keepdims=keepdims)
nan

So it appears that np.nanquantile calls np.nanmean. I was searching for nans but in fact an empty array passed to np.nanquantile caused the warning, and returned nan, causing nans to be passed to np.polyfit, which in turn caused the above error.

This answer helped me identify that the polyfit error occurred due to nans in the data.

bt3
  • 156
  • 1
  • 10