2

I'm using NumPy with Python 2.6.2. I'm trying to create a small (length 3), simple boolean array. The following gives me a MemoryError, which I think it ought not to.

import numpy as np
cond = np.fromiter((x in [2] for x in [0, 1, 2]), dtype = np.bool)

The error it gives me is:

MemoryError: cannot allocate array memory

However, the following method of obtaining a list (as opposed to an ndarray) works fine (without using numpy):

cond = list((x in [2] for x in [0, 1, 2]))

Have I done anything wrong in the Numpy code? My feeling is that it ought to work.

Underflow
  • 23
  • 1
  • 3
  • 1
    Your first code works for me as is. Can you post the versions of Python and Numpy? – Muhammad Alkarouri Sep 15 '10 at 12:13
  • For what it's worth, I can reproduce the problem using python 2.5 and numpy 1.1, but not with anything newer. On the older versions, it works fine if you manually specify the `count` kwarg (`count=3`, in this case). However, that defeats the purpose of using `np.fromiter` in the first place. I assmume it's a bug in numpy that's been fixed somewhere between 1.1 and 1.5? – Joe Kington Sep 15 '10 at 13:32

2 Answers2

1

You should not get any error.

With Python 2.6.5 or Python 2.7, and Numpy 1.5.0, I don't get any error. I therefore think that updating your software could very well solve the problem that you observe.

Eric O. Lebigot
  • 91,433
  • 48
  • 218
  • 260
1

I can reproduce the problem with numpy 1.1 (but not with anything newer). Obviously, upgrading to a more recent version of numpy is your best bet.

Nonetheless, it seems to be related to using np.bool as the dtype when count=-1 (the default: Read all items in the iterator, instead of a set number).

A quick workaround is just to create it as an int array and then convert it to a boolean array:

cond = np.fromiter((x in [2] for x in [0, 1, 2]), dtype=np.int).astype(np.bool)

An alternative is to convert it to a list, and then set count to the length of the list (or just use np.asarray on the list):

items = list((x in [2] for x in [0, 1, 2]))
cond = np.fromiter(items, dtype=np.bool, count=len(items))

Obviously, both of these are suboptimal, but if you can't upgrade to a more recent version of numpy, they will work.

Joe Kington
  • 275,208
  • 71
  • 604
  • 463