1

Note that nbytes doesn't provide the correct value. For instance:

>>> n = 1000
>>> x = np.arange(n)
>>> bx = np.broadcast_to(x, (int(1e15), n))
>>> bx.nbytes
8e18

....which probably requires more RAM than exists on Earth.

EDIT: More specifically, is there a way to obtain the size of the buffer that bx refers to? Something along the lines of:

>>> x.nbytes
8000
>>> bx.underlying_buffer_size()
8000
sophros
  • 14,672
  • 11
  • 46
  • 75
Mateen Ulhaq
  • 24,552
  • 19
  • 101
  • 135

1 Answers1

1

Note that as you can see in the docs, broadcast_to returns a view, where the broadcasted array may refer to a single memory location, from the docs:

broadcast : array A readonly view on the original array with the given shape. It is typically not contiguous. Furthermore, more than one element of a broadcasted array may refer to a single memory location.

Hence, in this case all new rows are pointing to the same memory location.

In order to see the actual of size of the object in bytes you can use sys.getsizeof:

from sys import getsizeof

getsizeof(bx)
112

This can be seen by checking which is the actual identity of the inner arrays:

id(bx[0])
# 1434315204368

id(bx[1])
# 1434315203968
yatu
  • 86,083
  • 12
  • 84
  • 139
  • 1
    See the doc: `All built-in objects will return correct results, but this does not have to hold true for third-party extensions as it is implementation specific. Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.`. 3rd party libs are free to implement this how they see fit and might account or not account for any big chunks of mem they hold (maybe through indirection). – Sebastian Hoffmann Feb 07 '20 at 12:05