4

This question is related to how to check the dtype of a column in python pandas.

An empty pandas dataframe is created. Following this, it's filled with data. How can I then check if any of its columns contain complex types? 

index = [np.array(['foo', 'qux'])]
columns = ["A",  "B"]
df = pd.DataFrame(index=index, columns=columns)
df.loc['foo']["A"] = 1 + 1j
df.loc['foo']["B"] = 1
df.loc['qux']["A"] = 2
df.loc['qux']["B"] = 2

print df
for type in df.dtypes:
    if type == complex:
        print type

At the moment, I get the type as object which isn't useful.

          A  B
foo  (1+1j)  1
qux       2  2
Community
  • 1
  • 1
bluprince13
  • 4,607
  • 12
  • 44
  • 91
  • You'd have to do `type(df['A'].iloc[0])` assuming all values are the same, the `np.object` dtype is used for python object representation – EdChum Feb 08 '17 at 10:10
  • You could do `isinstance(df.A.iloc[0], complex)`? – Zero Feb 08 '17 at 10:11
  • @EdChum in my case some of the values are real, and some complex! – bluprince13 Feb 08 '17 at 10:11
  • having mixed dtypes will be problematic, additionally if the types are not numpy friendly types you will find you will lose the performance gain of using pandas meaning that you will lose vectorised operations, all you will gain is easier indexing semantics – EdChum Feb 08 '17 at 10:13
  • 1
    `(1+1j)` is a plain Python `complex` object. `np.complex128(1+1j)` is a NumPy `complex128` object. You need to use values with native NumPy dtypes to get `df.dtypes` to report non-object dtypes. – unutbu Feb 08 '17 at 10:13

1 Answers1

2

Consider the series s

s = pd.Series([1, 3.4, 2 + 1j], dtype=np.object)
s

0         1
1       3.4
2    (2+1j)
dtype: object

If I use pd.to_numeric, it will upcast the dtype to complex if any are complex

pd.to_numeric(s).dtype

dtype('complex128')
piRSquared
  • 285,575
  • 57
  • 475
  • 624
  • For anyone else who stumbles across this, you can apply this to the whole dataframe with `df.apply(pd.to_numeric)` – bluprince13 Dec 08 '17 at 22:05