81

Since NumPy version 19.0, one must specify dtype=object when creating an array from "ragged" sequences. I'm faced with a large number of array calls from my own code and Pandas using threading, line-by-line debugging led me nowhere.

I'd like to figure out which call resulted in VisibleDeprecationWarning in my own code or a call from Pandas. How would I be able to debug this? I've been looking through the source and I cannot see this warning getting called in Python (only in numpy.core._multiarray_umath.cp38-win_amd64.pyd).

Ibu
  • 42,752
  • 13
  • 76
  • 103
misantroop
  • 2,276
  • 1
  • 16
  • 24

6 Answers6

89

With a function that creates a ragged array:

In [60]: def foo(): 
    ...:     print('one') 
    ...:     x = np.array([[1],[1,2]]) 
    ...:     return x 
    ...:                                                                                             
In [61]: foo()                                                                                       
one
/usr/local/bin/ipython3:3: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  # -*- coding: utf-8 -*-
Out[61]: array([list([1]), list([1, 2])], dtype=object)

I get the warning, but also the expected result.

I can control the warnings.

For example to turn if off:

In [68]: np.warnings.filterwarnings('ignore', category=np.VisibleDeprecationWarning)                 
In [69]: foo()                                                                                       
one
Out[69]: array([list([1]), list([1, 2])], dtype=object)

Or to raise an error:

In [70]: np.warnings.filterwarnings('error', category=np.VisibleDeprecationWarning)                  
In [71]: foo()                                                                                       
one
---------------------------------------------------------------------------
VisibleDeprecationWarning                 Traceback (most recent call last)
<ipython-input-71-c19b6d9633cf> in <module>
----> 1 foo()

<ipython-input-60-6ad21d9e07b4> in foo()
      1 def foo():
      2     print('one')
----> 3     x = np.array([[1],[1,2]])
      4     return x
      5 

VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray

The error gives a traceback telling me where the warning was raised.

There may be ways of refining the warning filter to catch just this one, and not others of the same category. I haven't used this mechanism much.

Read np.warnings.filterwarnings docs for more details.

hpaulj
  • 221,503
  • 14
  • 230
  • 353
  • This is an array of lists. How can I create array of arrays. That is for the same input lists, I want an output like this `[[1] [1 2]]` where `[1]` and `[1 2]` are arrays? Please reply. Thankyou! – Aman Kushwaha Sep 01 '21 at 06:56
  • @AmanKushwaha, `np.array([np.array([1]), np.array([1, 2])], dtype=object)` – hpaulj Sep 01 '21 at 07:19
  • @hpaulj after that, I receive value error : could not broadcast input array from shape... into.. Also, I am doing this thing for large number of arrays. – Aman Kushwaha Sep 01 '21 at 07:42
  • @AmanKushwaha, the example I gave does work! But there are some combinations of array shapes (e.g. 2d with same first dimension) that produce this kind of error. It's too big a topic to address in comments. And it's been addressed in other SO. But why are you trying to do this at all? – hpaulj Sep 01 '21 at 15:29
  • @hpaulj thank you very much for your reply. Actually I had n number of 2-D arrays each having same number of rows but different number of columns, I just wanted to make another arrray consisting of these 2-D arrays as elements, meaning I wanted to create a 3-D array, but I couldn't, then instead, I created a list, an empty list and appended these n arrays to the empty list using for loop. Now I want to convert this list (with n 2-D arrays as its elements) to an array(so that i get that 3-D array) but this command `np.array(list)` gave me value error: could not broadcast input array from shape – Aman Kushwaha Sep 01 '21 at 16:39
  • @AmanKushwaha, the most general way of creating an object dtype array is to initialize one with `x=np.empty((n,), object)`, and then assign the list of arrays, `x[:]=alist_of_n_arrays`. – hpaulj Sep 01 '21 at 17:46
  • What exactly makes a ragged nested sequence? – stefanbschneider Dec 10 '21 at 10:00
29
b2 = np.array(
    [[1, 2, 3, 4, 5, 6, 7, 8, 9, 10], [11, 12, 13, 14, 15, 16, 17, 18]],
    dtype=object,
)

Reference to the above example will clear the warning. You must specify dtype=object.

atharva
  • 80
  • 1
  • 7
Am_official
  • 467
  • 4
  • 5
7

This warning is caused by deprecated API of NumPy version 1.19 or higher, you may continue using it and just suppress the warning:

import warnings
warnings.filterwarnings("ignore", category=np.VisibleDeprecationWarning)
Waleed Aldhahi
  • 373
  • 5
  • 4
4

You can add dtype=object when you create your numpy array as :

numpy.array([[1,2,3],[4,5,6]], dtype=object)

or if you change a list or a tuple called 'a' to a numpy array code as:

numpy.asarray(a,dtype=object)

This helps you to avoid the warning.

user133639
  • 41
  • 1
2

I faced the np.VisibleDeprecationWarning while stacking lists containing audio data from WAV files. This problem occurred because audio files had different lengths. So, the lists that I needed to stack together into one numpy array also had varying lengths.

Ignoring or suppressing this warning did not give the desired stacked np array so, I made all the audio files of the same length by using pydub.AudioSegment as mentioned in this answer.

This resolved the warning.

Maria
  • 171
  • 1
  • 4
  • 1
    Ignoring the warning doesn't solve the issue. Your problem was specific to your case while the DeprecationWarning applies to a very large number of various cases. – misantroop May 06 '22 at 12:11
0

The length of elements in each lists should be the same. this warning was prompted due to the differences in the length of the arrays. arr3 = np.array([[5,6], [8, 9, 3]], dtype = object) specifying the dtype as objects increases the bytes used by each elements to 8 bytes. But without specify the dtype the memory size allocated is 4 bytes to each elements in the array.

DavidW
  • 29,336
  • 6
  • 55
  • 86
Brian
  • 1
  • 1