Formatting thousand separator for integers in a pandas dataframe

Question

I'm trying to use '{:,}'.format(number) like the example below to format a number in a pandas dataframe:

# This works for floats and integers
print '{:,}'.format(20000)
# 20,000
print '{:,}'.format(20000.0)
# 20,000.0

The problem is that with a dataframe that has integers does not work, and in a dataframe with float works ok. See the examples:

# Does not work. The format stays the same, does not show thousands separator
df_int = DataFrame({"A": [20000, 10000]})
print df_int.to_html(float_format=lambda x: '{:,}'.format(x))

# Example of result
# <tr>
#   <th>0</th>
#   <td> 20000</td>
# </tr

# Works OK
df_float = DataFrame({"A": [20000.0, 10000.0]})
print df_float.to_html(float_format=lambda x: '{:,}'.format(x))

# Example of result
# <tr>
#   <th>0</th>
#   <td>20,000.0</td>
# </tr>

What i'm doing wrong?

I'm not sure why it wouldn't work for integers, but can't you use a float and specify the precision, as in `{:,.0f}`? — dmvianna, Jul 23 '14 at 23:32
you need to specify a separate formatter for int; see [this](https://stackoverflow.com/questions/18876022/how-to-format-ipython-html-display-of-pandas-dataframe) question for example. — behzad.nouri, Jul 23 '14 at 23:37
But in that question he used the same format that I used `int_frmt = lambda x: '{:,}'.format( x )` — Javier Cárdenas, Jul 24 '14 at 01:15

score 14 · Answer 1 · edited Nov 02 '22 at 06:45

pandas (as of 0.20.1) does not allow overriding the default integer format in an easy way. It is hard coded in pandas.io.formats.format.IntArrayFormatter (the lambda function):

class IntArrayFormatter(GenericArrayFormatter):

    def _format_strings(self):
        formatter = self.formatter or (lambda x: '% d' % x)
        fmt_values = [formatter(x) for x in self.values]
        return fmt_values

I'm assuming what you're actually asking for is how you can override the format for all integers: modify (i.e. "monkey patch") the IntArrayFormatter to print integer values with thousands separated by comma as follows:

import pandas

class _IntArrayFormatter(pandas.io.formats.format.GenericArrayFormatter):

    def _format_strings(self):
        formatter = self.formatter or (lambda x: ' {:,}'.format(x))
        fmt_values = [formatter(x) for x in self.values]
        return fmt_values

pandas.io.formats.format.IntArrayFormatter = _IntArrayFormatter

Note:

before 0.20.0, the formatters were in pandas.formats.format.
before 0.18.1, the formatters were in pandas.core.format.

Aside

For floats you do not need to jump through those hoops since there is a configuration option for it:

display.float_format: The callable should accept a floating point number and return a string with the desired format of the number. This is used in some places like SeriesFormatter. See core.format.EngFormatter for an example.

score 8 · Accepted Answer · answered Jul 24 '14 at 11:41

The formatters parameter in to_html will take a dictionary of column names mapped to a formatting function. Below has an example of a function to build a dict that maps the same function to both floats and ints.

In [250]: num_format = lambda x: '{:,}'.format(x)

In [246]: def build_formatters(df, format):
     ...:     return {column:format 
     ...:               for (column, dtype) in df.dtypes.iteritems()
     ...:               if dtype in [np.dtype('int64'), np.dtype('float64')]}
     ...: 

In [247]: formatters = build_formatters(df_int, num_format)


In [249]: print df_int.to_html(formatters=formatters)
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>A</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>20,000</td>
    </tr>
    <tr>
      <th>1</th>
      <td>10,000</td>
    </tr>
  </tbody>
</table>

score 1 · Answer 3 · answered Jul 16 '20 at 13:08

You can always cast your table to float64 and then use float_format as you like, especially if you are constructing a small table for viewing purposes. Instead of dealing with ints and floats separately this gives a quick solution.

df.astype('float64',errors='ignore').to_html(float_format=lambda x: format(x,',.2f'))

errors='ignore' is there to prevent raising an exception when a column can not be converted to floats, like strings.

Formatting thousand separator for integers in a pandas dataframe

3 Answers3

Aside

Linked

Related