Permute list of lists with mixed elements (np.random.permutation() fails with ValueError)

Question

I'm trying to permute a list composed of sublists with mixed-type elements:

import numpy as np

a0 = ['122', 877.503017, 955.471176, [21.701201, 1.315585]]
a1 = ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]]
a2 = ['177', 1038.686843, 1018.987868, [19.539959, 1.183997]]
a3 = ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]]

a = [a0, a1, a2, a3]

b = np.random.permutation(a)

This will fail with:

ValueError: cannot set an array element with a sequence

Is there a built in function that will allow me to generate such permutation?

I need to generate a single random permutation, I'm not trying to obtain all the possible permutations.

I checked the three answers given with:

import time
import random

# np.random.permutation()
start = time.time()
for _ in np.arange(100000):
    b = np.random.permutation([np.array(i, dtype='object') for i in a])
print(time.time() - start)

# np.random.shuffle()
start = time.time()
for _ in np.arange(100000):
    b = a[:]
    np.random.shuffle(b)
print(time.time() - start)

# random.shuffle()
start = time.time()
for _ in np.arange(100000):
    random.shuffle(a)
print(time.time() - start)

The results are:

1.47580695152
0.11471414566
0.26300907135

so the np.random.shuffle() solution is about 10x faster than np.random.permutation() and 2x faster than random.shuffle().

are you trying to generate a random permutation of the list, or to get all the permutations? — glS, Sep 08 '16 at 19:36
Just a single random permutation. I'll add this info to the question. — Gabriel, Sep 08 '16 at 19:37
This is a really weird way to use NumPy. You're going to have to deal with a lot of awkwardness and inefficiency if you want to try to structure your data like this with NumPy. — user2357112, Sep 08 '16 at 19:59
That very well could be the case. Perhaps it is more efficient to just use the `random` package as suggested below. — Gabriel, Sep 08 '16 at 20:00

score 2 · Accepted Answer · edited May 23 '17 at 11:51

2

What about using np.random.shuffle?

# if you want the result in another list, otherwise just apply shuffle to a
b = a[:]
# shuffle the elements
np.random.shuffle(b)
# see the result of the shuffling
print(b)

See this answer for the difference between shuffle and permutation

edited May 23 '17 at 11:51

Community

1
1

answered Sep 08 '16 at 19:43

glS

1,209
6
18
45

This returns `None` in my case? o_O – Gabriel Sep 08 '16 at 19:45
@Gabriel `shuffle` shuffles the array in-place. Print `b` to see the result. – glS Sep 08 '16 at 19:46
Of course. This looks even simpler than Kasramvd's answer. Perhaps I'd need to do some timing test to see which one is more efficient. – Gabriel Sep 08 '16 at 19:48
1

as highlighted in the linked question, `permutation` actually *uses* shuffle, as you can see from the [source code](https://github.com/numpy/numpy/blob/master/numpy/random/mtrand/mtrand.pyx#L5134), so you will probably find `shuffle` to be more efficient (or more likely, for the difference to be negligible) – glS Sep 08 '16 at 19:54

score 2 · Answer 2 · answered Sep 08 '16 at 19:56

If you just want to create a random permutation of a = [a0, a1, a2, a3], might I suggest permuting the indices instead?

>>> random_indices = np.random.permutation(np.arange(len(a)))
>>> a_perm = [a[i] for i in random_indices]
... # Or just use the indices as you see fit...

If you're using numpy just for this, skip numpy altogether instead and just use random.shuffle to effect the same:

>>> import random
>>> random.shuffle(a)

Mazdak · Answer 3 · 2016-09-08T19:43:28.120

You need to convert your list to numpy arrays with with type object(), so that random.permutation() can interpret the lists as numpy types rather than sequence:

>>> a = [np.array(i, dtype='object') for i in a]
>>> 
>>> np.random.permutation(a)
array([['122', 877.503017, 955.471176, [21.701201, 1.315585]],
       ['177', 1038.686843, 1018.987868, [19.539959, 1.183997]],
       ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]],
       ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]]], dtype=object)

You can also use create a uniqe array from your lists using numpy.array() instead of using a list comprehension:

>>> a = np.array((a0, a1, a2, a3), dtype='object')
>>> a
array([['122', 877.503017, 955.471176, [21.701201, 1.315585]],
       ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]],
       ['177', 1038.686843, 1018.987868, [19.539959, 1.183997]],
       ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]]], dtype=object)
>>> np.random.permutation(a)
array([['122', 877.503017, 955.471176, [21.701201, 1.315585]],
       ['177', 1038.686843, 1018.987868, [19.539959, 1.183997]],
       ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]],
       ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]]], dtype=object)
>>> np.random.permutation(a)
array([['177', 1038.686843, 1018.987868, [19.539959, 1.183997]],
       ['176', 1134.076908, 1125.504758, [19.436181, 0.9987899]],
       ['178', 878.999081, 1022.050447, [19.6448771, 1.1867719]],
       ['122', 877.503017, 955.471176, [21.701201, 1.315585]]], dtype=object)

score 0 · Answer 4 · answered Sep 08 '16 at 19:50

0

random.shuffle() changes the list in place.

Python API methods that alter a structure in-place generally return None.

Please try random.sample(a,len(a))

The code would look like:

a = a[:]
b = random.sample(a,len(a))

answered Sep 08 '16 at 19:50

v.coder

1,822
2
15
24

Permute list of lists with mixed elements (np.random.permutation() fails with ValueError)

4 Answers4