How do I find numeric columns in Pandas?

Question

Let's say df is a pandas DataFrame. I would like to find all columns of numeric type. Something like:

isNumeric = is_numeric(df)

Answer 1

You could use select_dtypes method of DataFrame. It includes two parameters include and exclude. So isNumeric would look like:

numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']

newdf = df.select_dtypes(include=numerics)

Answer 2

Simple one-line answer to create a new dataframe with only numeric columns:

df.select_dtypes(include=np.number)

If you want the names of numeric columns:

df.select_dtypes(include=np.number).columns.tolist()

Complete code:

import pandas as pd
import numpy as np

df = pd.DataFrame({'A': range(7, 10),
                   'B': np.random.rand(3),
                   'C': ['foo','bar','baz'],
                   'D': ['who','what','when']})
df
#    A         B    C     D
# 0  7  0.704021  foo   who
# 1  8  0.264025  bar  what
# 2  9  0.230671  baz  when

df_numerics_only = df.select_dtypes(include=np.number)
df_numerics_only
#    A         B
# 0  7  0.704021
# 1  8  0.264025
# 2  9  0.230671

colnames_numerics_only = df.select_dtypes(include=np.number).columns.tolist()
colnames_numerics_only
# ['A', 'B']

Answer 3

You can use the undocumented function _get_numeric_data() to filter only numeric columns:

df._get_numeric_data()

Example:

In [32]: data
Out[32]:
   A  B
0  1  s
1  2  s
2  3  s
3  4  s

In [33]: data._get_numeric_data()
Out[33]:
   A
0  1
1  2
2  3
3  4

Note that this is a "private method" (i.e., an implementation detail) and is subject to change or total removal in the future. Use with caution.

Answer 4

df.select_dtypes(exclude = ['object'])

Update:

df.select_dtypes(include= np.number)

or with new version of panda

 df.select_dtypes('number')

Answer 5

Simple one-liner:

df.select_dtypes('number').columns

Answer 6

Following codes will return list of names of the numeric columns of a data set.

cnames=list(marketing_train.select_dtypes(exclude=['object']).columns)

here marketing_train is my data set and select_dtypes() is function to select data types using exclude and include arguments and columns is used to fetch the column name of data set output of above code will be following:

['custAge',
     'campaign',
     'pdays',
     'previous',
     'emp.var.rate',
     'cons.price.idx',
     'cons.conf.idx',
     'euribor3m',
     'nr.employed',
     'pmonths',
     'pastEmail']
    

Answer 7

This is another simple code for finding numeric column in pandas data frame,

numeric_clmns = df.dtypes[df.dtypes != "object"].index 

Answer 8

We can include and exclude data types as per the requirement as below:

train.select_dtypes(include=None, exclude=None)
train.select_dtypes(include='number') #will include all the numeric types

Referred from Jupyter Notebook.

To select all numeric types, use np.number or 'number'

• To select strings you must use the object dtype but note that this will return all object dtype columns

• See the NumPy dtype hierarchy <http://docs.scipy.org/doc/numpy/reference/arrays.scalars.html>__

• To select datetimes, use np.datetime64, 'datetime' or 'datetime64'

• To select timedeltas, use np.timedelta64, 'timedelta' or 'timedelta64'

• To select Pandas categorical dtypes, use 'category'

• To select Pandas datetimetz dtypes, use 'datetimetz' (new in 0.20.0) or ``'datetime64[ns, tz]'

Answer 9

Although this is old subject,

but i think the following formula is easier than all other comments

df[df.describe().columns]

As the function describe() only works for numeric columns, the column of the output will only be numeric.

Answer 10

Please see the below code:

if(dataset.select_dtypes(include=[np.number]).shape[1] > 0):
display(dataset.select_dtypes(include=[np.number]).describe())
if(dataset.select_dtypes(include=[np.object]).shape[1] > 0):
display(dataset.select_dtypes(include=[np.object]).describe())

This way you can check whether the value are numeric such as float and int or the srting values. the second if statement is used for checking the string values which is referred by the object.

Answer 11

Adapting this answer, you could do

df.ix[:,df.applymap(np.isreal).all(axis=0)]

Here, np.applymap(np.isreal) shows whether every cell in the data frame is numeric, and .axis(all=0) checks if all values in a column are True and returns a series of Booleans that can be used to index the desired columns.

Answer 12

A lot of the posted answers are inefficient. These answers either return/select a subset of the original dataframe (a needless copy) or perform needless computational statistics in the case of describe().

To just get the column names that are numeric, one can use a conditional list comprehension with the pd.api.types.is_numeric_dtype function:

numeric_cols = [col for col in df if pd.api.types.is_numeric_dtype(df[col])]

I'm not sure when this function was introduced.

Answer 13

@Kathiramani Sukumar's answer df._get_numeric_data() takes the cake.

xdf = pd.DataFrame({'Numeric':[20,10,np.nan],'String':['foo','bar','daa'],'Date':[datetime(2023,1,1,0,0,0),datetime(2023,1,2,0,0,0),np.nan]})
xdf.dtypes
Numeric           float64
String             object
Date       datetime64[ns]
dtype: object

%timeit xdf._get_numeric_data()
34.7 µs ± 870 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

%timeit xdf.select_dtypes(include=np.number)
797 µs ± 10.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']
%timeit xdf.select_dtypes(include=numerics)
991 µs ± 24.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Answer 14

def is_type(df, baseType):
    import numpy as np
    import pandas as pd
    test = [issubclass(np.dtype(d).type, baseType) for d in df.dtypes]
    return pd.DataFrame(data = test, index = df.columns, columns = ["test"])
def is_float(df):
    import numpy as np
    return is_type(df, np.float)
def is_number(df):
    import numpy as np
    return is_type(df, np.number)
def is_integer(df):
    import numpy as np
    return is_type(df, np.integer)

Answer 15

numerical_col = df.describe().columns.to_list()

This is what i normally use. Since the describe method only returns numerical columns.