1

1.I use cv2.imread to read a big image in numpy array (1234*1624*3)

2.I use cv2.dnn.blobFromImage to transform it to (1,3,1234,1624) numpy array

3.I use tolist() to transform it to a 4D list in lists

My Problem:

How to transpose this list's axis

from (1,3,1234,1624) to (1,1234,1624,3)?

I tried np.transpose(image, (0, 2, 3, 1)) first then do the tolist(), but it appears to be very slow. Therefore, I want to transpose in the list form.

I've checked the question below it seems fast but can only transpose 2D list Transpose list of lists

Much appreciated!!

Update:

I have a list named temp.

len(temp)=1 len(temp[0])=3 len(temp[0][0])=1234 len(temp[0][0][0])=1624

and I want to transpose its axis to

len(temp)=1 len(temp[0])=1234 len(temp[0][0])=1624 len(temp[0][0][0])=3

How to do with temp1=list(map(list, zip(*temp))?

Bill Huang
  • 53
  • 1
  • 7
  • Use the idiomatic expression `zip(*zip(*items))` – norok2 Jul 03 '19 at 06:31
  • I tried `temp1=list(map(list, zip(zip(*temp))))` but doesn't work for me – Bill Huang Jul 03 '19 at 06:46
  • Sorry I understood it wrongly, the double zip trick work for 1-level nesting iterators only. I think NumPy is the fastest. Why do you need to convert it to `list`? – norok2 Jul 03 '19 at 06:48
  • I need to send the request in Json format to Tensorflow serving. I've found `ndnumpy (1,3,1234,1624).tolist()` is much faster than `ndnumpy (1,1234,1624,3).tolist()` – Bill Huang Jul 03 '19 at 06:52
  • (1,3,1234,1624) is the shape of ndnumpy array – Bill Huang Jul 03 '19 at 06:53
  • That array transpose, `np.transpose(image, (0, 2, 3, 1))` should be fast, since it makes a view, not a copy. Doing a `tolist()` on the may be a bit slower (than a `tolist` on `image`), because it probably involves a copy. But it should be easier and faster than trying to do the equivalent transpose with lists. – hpaulj Jul 03 '19 at 07:14
  • I'm familiar with the `zip*` idiom for transposing a simple nested list, I'd have experiment to come up with an equivalent for this 4d nesting (3d actually since we can ignore the first size one dimension). Even with that I'd probably be testing my results against the array version, which is easy to understand. – hpaulj Jul 03 '19 at 07:17
  • `np.transpose(image, (0, 2, 3, 1))` is very fast. However, Doing a `tolist()` on a `(1,1234,1624,3)` numpy array takes 5 time more than a `tolist()` on a `(1,3,1234,1624)` numpy array on my machine. – Bill Huang Jul 03 '19 at 07:19
  • Making a nested list with that small size 3 inner most will be costly how ever you do it. You'll have a million size 3 liats, and pointers to each. – hpaulj Jul 03 '19 at 07:46
  • @hpaulj do you mind sharing your code? Thanks a lot! – Bill Huang Jul 03 '19 at 07:52

1 Answers1

1

It can be done numpy-less and purely functional but it's not pretty:

from itertools import starmap, repeat

a = np.ones((1,3,1234,1624)).tolist()
b = list(map(list, map(map, repeat(list), map(starmap, repeat(zip), starmap(zip, a)))))
np.shape(b)
# (1, 1234, 1624, 3)
Paul Panzer
  • 51,835
  • 3
  • 54
  • 99
  • Thanks for your reply! `np.ones((1,3,1234,1624)).tolist()`+`list(map(list, map(map, repeat(list), map(starmap, repeat(zip), starmap(zip, a)))))` takes only half of the time of `np.transpose(image, (0, 2, 3, 1))`+`tolist()` on my machine! That's a lot! – Bill Huang Jul 03 '19 at 10:04
  • However, I'm a little confused about how does those map and zip works. Is it possible for you to simply explain if i want to generalize to other shape N-D list? – Bill Huang Jul 03 '19 at 10:30
  • @BillHuang I'm afraid this is about as far as I am able to push it. I don't think it generalizes well (expressions just become longer and longer). As for explanation. We can only swap adjacent axes with `zip`, so we have to construct the shuffle we want from such swaps. The swaps look like: swap axes 0 and 1: zip(*a) ; swap axes 1 and 2: starmap(zip, a) ; swap axes 2 and 3: map(starmap, repeat(zip), a) etc. In Python3 all those return generators, so we have to force lists in an additional step. – Paul Panzer Jul 03 '19 at 10:52
  • So begining --> `(1, 3, 1234, 1624)` Next,`starmap(zip, a)` --> `(1, 1234, 3, 1624)` (swap axes 1 and 2) Next,`map(starmap, repeat(zip), starmap(zip, a)))`-->`(1, 1234, 1624, 3)`(swap axes 2 and 3) Am I understanding this correctly? Then what's `map(list, map(map, repeat(list), )` doing outside? Thank you very much for your reply! @PaulPanzer – Bill Huang Jul 04 '19 at 01:02
  • @BillHuang yes, that's correct so far. As I said, these do not create lists but generators. A generator is something like a "suspended for loop", you have to loop through it or map it to a list or tuple, otherwise it does nothing. As with the swaps, these generators are created at different depths of nesting, which is why we have to administer the `list` cast through `map`s and nested `map`s. – Paul Panzer Jul 04 '19 at 01:57