3

I want to create a 3D array in Python, filled with -1.

I tested these methods:

import numpy as np
l = 200
b = 100
h = 30

%timeit grid = [[[-1 for x in range(l)] for y in range(b)] for z in range(h)]
1 loops, best of 3: 458 ms per loop

%timeit grid = -1 * np.ones((l, b, h), dtype=np.int)
10 loops, best of 3: 35.5 ms per loop

%timeit grid = np.zeros((l, b, h), dtype=np.int) - 1
10 loops, best of 3: 31.7 ms per loop

%timeit grid = -1.0 * np.ones((l, b, h), dtype=np.float32)
10 loops, best of 3: 42.1 ms per loop

%%timeit
grid = np.empty((l,b,h))
grid.fill(-1.0)
100 loops, best of 3: 13.7 ms per loop

So obviously, the last one is the fastest. Does anybody has an even faster method or at least less memory intensive? Because it runs on a RaspberryPi.

jonrsharpe
  • 115,751
  • 26
  • 228
  • 437
Balzer82
  • 999
  • 4
  • 10
  • 21
  • I think the most common way to create such an array is: `grid = np.full((l, b, h), -1)`. Not faster though. – Rutger Kassies Sep 23 '14 at 10:35
  • Oh, I saw, my question is a dup of [this](http://stackoverflow.com/questions/5891410/numpy-array-initialization-fill-with-identical-values). – Balzer82 Sep 23 '14 at 10:47

2 Answers2

3

The only thing I can think to add is that any of these methods will be faster with the dtype argument chosen to take up as little memory as possible.

Assuming you need no more space that int8, the method suggested by @RutgerKassies in the comments took this long on my system:

%timeit grid = np.full((l, b, h), -1, dtype=np.int8)
1000 loops, best of 3: 286 µs per loop

For comparison, not specifying dtype (defaulting to int32) took about 10 times longer with the same method:

%timeit grid = np.full((l, b, h), -1)
100 loops, best of 3: 3.61 ms per loop

Your fastest method was about as fast as np.full (sometimes beating it):

%%timeit
grid = np.empty((l,b,h))
grid.fill(-1)
100 loops, best of 3: 3.51 ms per loop

or, with dtype specified as int8,

1000 loops, best of 3: 255 µs per loop

Edit: This is probably cheating, but, well...

%timeit grid = np.lib.stride_tricks.as_strided(np.array(-1, dtype=np.int8), (l, b, h), (0, 0, 0))
100000 loops, best of 3: 12.4 us per loop

All that's happening here is that we begin with an array of length one, np.array([-1]), and then fiddle with the stride lengths so that grid looks exactly like an array with the required dimensions.

If you need an actual array, you can use grid = grid.copy(); this makes the creation of the grid array about as fast as the quickest approaches suggested on elsewhere this page.

mathfux
  • 5,759
  • 1
  • 14
  • 34
Alex Riley
  • 169,130
  • 45
  • 262
  • 238
  • Looking at the code for `np.full` - it creates an `empty`, and uses `copyto` broadcast the value to array. `x[:]=...` quite possibly uses the same underlying C code. – hpaulj Sep 23 '14 at 20:22
0

This is a little faster for me. Might be different on a RPi though.

grid = np.empty((l,b,h))
grid[...] = -1

np.int8 is much faster if it's big enough for you

grid = np.empty((l,b,h), dtype=np.int8)
grid[...] = -1

%%timeit 
grid = np.empty((l,b,h), dtype=np.int8)
grid[:] = -1 
100 loops, best of 3: 6.91 ms per loop
John La Rooy
  • 295,403
  • 53
  • 369
  • 502
  • ```%%timeit grid = np.empty((l,b,h), dtype=np.int8) grid[:] = -1 100 loops, best of 3: 6.91 ms per loop``` – Balzer82 Sep 23 '14 at 11:59