Python : conversion from numpy.ndarray to list : getting hung

Question

I have a list named word_vector whose every element is of type 'numpy.ndarray'.

print(word_vector)

output:

[array([0., 0., 0., ..., 0., 0., 0.]), array([0., 0., 0., ..., 0., 0., 0.]), array([0., 0., 0., ..., 0., 0., 0.]), array([0., 0., 0., ..., 0., 0., 0.])]

I want to convert the type of each element of the list to list. So I wrote this code:

word_vector_list = []
for arr in word_vector:
    list_ = arr.tolist()
    word_vector_list.append(list_)
    
print(word_vector_list)

The programming is getting hung repeatedly. I'm getting this exception:

IOPub data rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
`--NotebookApp.iopub_data_rate_limit`.

Current values:
NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec)
NotebookApp.rate_limit_window=3.0 (secs)

length of each element(of type array) of the list is:

print(len(arr))

output:

From the above data, the size seems to be just 4 x 4395. Have you tried converting just one nd array from the list and see how long does it take? — Venkata Rathnam, Jun 21 '20 at 12:26
just tried for one element. It didn't get hung, but I've 10k * 4395 data. — Debbie, Jun 21 '20 at 12:31

score 1 · Answer 1 · answered Jun 21 '20 at 12:29

1

I'm not entirely sure why your program is hanging. Try using list comprehension like so, it should be better:

word_vector_list = [list(x) for x in word_vector]

If it still didn't work, try increasing the iopub data rate. To increase the iopub data rate, run this in the terminal:

jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10

Hope this helps :)

answered Jun 21 '20 at 12:29

Sahith Kurapati

1,617
10
14

Problem persists even after increasing the data rate. – Debbie Jun 21 '20 at 13:09

score 0 · Answer 2 · answered Jun 21 '20 at 13:16

0

2d np array can be changed to list of lists directly by .tolist function.

Also seen here

Thanks

answered Jun 21 '20 at 13:16

Rupesh

530
2
14

Python : conversion from numpy.ndarray to list : getting hung

2 Answers2