7

I am receiving a warning and I want to check if this will break. I am using np.where like this in a lot of cases (it is similar, for me, to an if statement in excel). Is there a better or more pythonic or pandas way to do this? I'm trying to turn one dimension into something I can easily do mathematical operations on. 

df['closed_item'] = np.where(df['result']=='Action Taken', 1, 0)

FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  result = getattr(x, name)(y)


INSTALLED VERSIONS
------------------
python: 3.5.1.final.0
python-bits: 64
OS: Windows
OS-release: 10

pandas: 0.18.0
nose: 1.3.7
pip: 8.1.0
setuptools: 20.2.2
Cython: 0.23.4
numpy: 1.11.0
scipy: 0.17.0
statsmodels: 0.6.1
xarray: None
IPython: 4.0.0
sphinx: 1.3.1
patsy: 0.4.0
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: None
tables: 3.2.2
numexpr: 2.5.1
matplotlib: 1.5.1
openpyxl: 2.2.6
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.7.7
lxml: 3.4.4
bs4: 4.4.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.9
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.38.0
trench
  • 5,075
  • 12
  • 50
  • 80
  • What is your version of `pandas` and `numpy`? `print pd.show_versions()` – jezrael Apr 14 '16 at 10:27
  • Does this answer your question? [numpy.where() function throws a FutureWarning, returns scalar instead of list](https://stackoverflow.com/questions/40659212/futurewarning-elementwise-comparison-failed-returning-scalar-but-in-the-futur) – usr Dec 04 '19 at 15:20
  • Does this answer your question? [FutureWarning: elementwise comparison failed; returning scalar, but in the future will perform elementwise comparison](https://stackoverflow.com/questions/40659212/futurewarning-elementwise-comparison-failed-returning-scalar-but-in-the-futur) – Grzegorz Skibinski Jan 08 '20 at 18:16

2 Answers2

3

This warning occurs when comparing "int" and "str" in your dataset. Add .astype(int) to your comparison dataset. Try:

df['closed_item'] = np.where(df['result'].astype(str)=='Action Taken', 1, 0)
Allen Sun
  • 178
  • 1
  • 4
  • 9
1

The issue that you mentioned is actually quite complex, so let me divide it into parts using your words:

I am receiving a warning and I want to check if this will break

A Warning is a statement that is telling you to be cautious with how you handle your coding logic. A well-designed warning is not going to break your code; if it were a case, it would be an Exception.

While you need to be concerned if there are problems with your output or performance, often you may ignore a warning ceteris paribus. So in your case, if everything else is OK and you do not plan to update the software, you do not need to do anything to suppress the warning. However, if you need to, you may use the following snippet:

import warnings
with warnings.catch_warnings():
    warnings.filterwarnings('ignore', r'elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison(.*)')

I am using np.where like this in a lot of cases (it is similar, for me, to an if statement in excel).

Note that there is a DataFrame.where method in pandas.

Is there a better or more pythonic or pandas way to do this?

Yes, there are two ways that you can use to make your code more pandas-like: If you want to get a number of columns that will work like dummies, you may use 

pd.get_dummies(df.result)

It will produce a data frame with all possible dummy values it could find in a series. If this sounds to you like an overkill, do not worry there are ways to single out just one such variable.

In pandas boolean True and False are commonly used to binary classify matches within a series or a dataframe so in your case, one could perform the following operation:

df.closed_item = df.result == 'Action Taken'

I'm trying to turn one dimension into something I can easily do mathematical operations on.

However, if you want the output to contain integer values so that it matches yours, you may use this piece of code:

df.closed_item = (df.result == 'Action Taken'`).astype(int)

As a side note, I do not think this warning propagates to newer versions, i.e. 0.13 and above (as expected since it is a future warning), so you may also considering an update.

Arn
  • 1,898
  • 12
  • 26
  • I also can't reproduce the issue, I would think you might be wrong about the reason in here- as per one of the comments- this seems to be more appropriate answer in here: https://stackoverflow.com/a/46721064/11610186 – Grzegorz Skibinski Jan 08 '20 at 18:15