6

I ran into a memory problem when trying to use .reshape on a numpy array and figured if I could somehow reshape the array in place that would be great.

I realised that I could reshape arrays by simply changing the .shape value. Unfortunately when I tried using .shape I again got a memory error which has me thinking that it doesn't reshape in place.

I was wondering when do I use one when do I use the other?

Any help is appreciated.

If you want additional information please let me know.

EDIT:

I added my code and how the matrix I want to reshape is created in case that is important.

Change the N value depending on your memory.

import numpy as np
N = 100
a = np.random.rand(N, N)
b = np.random.rand(N, N)
c = a[:, np.newaxis, :, np.newaxis] * b[np.newaxis, :, np.newaxis, :]
c = c.reshape([N*N, N*N])
c.shape = ([N, N, N, N])

EDIT2: This is a better representation. Apparently the transpose seems to be important as it changes the arrays from C-contiguous to F-contiguous, and the resulting multiplication in above case is contiguous while in the one below it is not.

import numpy as np
N = 100
a = np.random.rand(N, N).T
b = np.random.rand(N, N).T
c = a[:, np.newaxis, :, np.newaxis] * b[np.newaxis, :, np.newaxis, :]
c = c.reshape([N*N, N*N])
c.shape = ([N, N, N, N])
evan54
  • 3,585
  • 5
  • 34
  • 61

2 Answers2

8

numpy.reshape will copy the data if it can't make a proper view, whereas setting the shape will raise an error instead of copying the data.

It is not always possible to change the shape of an array without copying the data. If you want an error to be raise if the data is copied, you should assign the new shape to the shape attribute of the array.

ryanpattison
  • 6,151
  • 1
  • 21
  • 28
  • So the behaviour is exactly the same except how they handle the need for a copy? Also when would it need to be copied? – evan54 Nov 11 '14 at 02:28
  • @evan54 If the array is not contiguous it cannot be reshaped in-place, see the comments in the answer to [reshape an array in numpy](http://stackoverflow.com/questions/14476415/reshape-an-array-in-numpy) – ryanpattison Nov 11 '14 at 02:37
0

I would like to revisit this question focusing on OOP paradigm, despite memory issues presented as the problem.

When to use .shape and when to use .reshape?

OOP principle of Encapsulation

Following OOP paradigms, since shape is a property of the object numpy.array it is always advisable to call an object.method to change properties. This adheres to OOP principle of encapsulation.

Performance Issues

As for performance, there seems to be no difference.

import numpy as np
# creates an array of 1,000,000 random floats
a = np.array(np.random.rand(1_000_000))

# (1000000,)
a.shape                   

# using IPython to time both operations resulted in

# 201 ns ± 4.85 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
%timeit a.shape = (5_000, 200)

# 217 ns ± 0.957 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit a.reshape (5_000, 200)

Running hardware

OS : Linux 4.15.0-142-generic #146~16.04.1-Ubuntu CPU: Intel(R) Core(TM) i3-4170 CPU @ 3.70GHz 4 cores RAM: 16BG

Jayr Magave
  • 78
  • 8