5

I have pandas table with two columns with numerical data (dtype flaot64). I have rounded each column to have 2 digits after the decimal point and then used function to round it to the near 0.5 but for some reason only one column got rounded with 0.05 and the second one got rounded but missed the 2nd digit.

This is fake example which works and show the flow :

table=pd.DataFrame({'A': [0.62435, 0.542345,0.213452],
                   'B': [0.22426,0.15779,0.30346]})

#function for round to near 0.5:
def custom_round(x, base=5):
    return base * round(float(x)/base)

table['A'] = table['A'].astype(float).round(2).apply(lambda x: custom_round(x, base=.05))
table['B'] = table['B'].astype(float).round(2).apply(lambda x: custom_round(x, base=.05))
table

>>>

A   B
0   0.60    0.20
1   0.55    0.15
2   0.20    0.30

but on my table I get in the end:

enter image description here

When I run the script without the function to round near 0.5, I still get the two digits:

table['B'] = table['B'].round(2)

enter image description here

My question is why is this hapenning? and how can I fix it in order to round both columns to 0.05 and get both digits appear?

edit: I have been asked how do I apply it on my real table , so:

df['A'] = df['A'].astype(float).round(2).apply(lambda x: custom_round(x, base=.05))
df['B']= df['B'].round(2).apply(lambda x: custom_round(x, base=.05))
Vishesh Mangla
  • 664
  • 9
  • 20
Reut
  • 1,555
  • 4
  • 23
  • 55
  • Hey I just ran this character for character and it seems to work as you expect.Is there any thing else also happening in the environment where you are running this, have you done any additional operations above or below this snippet of code where you actually see the output that's not as per your expectations? – lytseeker Jul 09 '20 at 08:51
  • Is there any difference on how you apply the function on the 'real' table? – Shaido Jul 09 '20 at 08:55
  • 1
    @Shaido-ReinstateMonica I have edited my post . just to add that I have tried to do the astype float for both, also float64 but got the same results – Reut Jul 09 '20 at 09:01
  • Instead of posting an example "that works", you should post an example that does not work. – Christian Fries Jul 11 '20 at 12:54
  • What's the output of `print(pd.get_option("display.precision"))` for you? (see for example the comment below [this answer](https://stackoverflow.com/a/44963865/565489) ) – Asmus Jul 16 '20 at 19:23

3 Answers3

4

Your numbers are rounded correctly. Below I will explain,

  1. How to show 2 digits precision?
  2. What was happening with the example data?

1. How to show 2 digits precision?

If you really want just to show two digits, you can skip the rounding function (custom_round) altogether, and just run this* before printing your dataframes:

pd.options.display.float_format = '{:,.2f}'.format

This will make the float valued data to be printed with 2 digits precision. Example:

table=pd.DataFrame({'A': [0.62435, 0.542345,0.213452],
                   'B': [0.22426,0.18779,0.30346]})
In [1]: table
Out[1]:
     A    B
0 0.62 0.22
1 0.54 0.19
2 0.21 0.30

2. What is happening with the example data?

  • Using the same data as given in the question
table=pd.DataFrame({'A': [0.62435, 0.542345,0.213452],
                   'B': [0.22426,0.15779,0.30346]})

# execute code with custom_round in the question

In [1]: table
Out[1]:
      A     B
0  0.60  0.20
1  0.55  0.15
2  0.20  0.30
  • Setting the middle value of B to 0.18779 (rounded to 0.20)
table=pd.DataFrame({'A': [0.62435, 0.542345,0.213452],
                   'B': [0.22426,0.18779,0.30346]})

# execute code with custom_round in the question

In [1]: table
Out[1]:
      A    B
0  0.60  0.2
1  0.55  0.2
2  0.20  0.3

Why does this happen?

Internally, the number is rounded into two digit precision. When you print the table to the console / Jupyter notebook, pandas skips printing of the last value (2nd digit) if they are all zeroes. So, the data is two digits precision (for example, 0.20), but it is just shown with one digit precision, since 0.20 = 0.2.

* You may also use other printing scheme: The pd.options.display.float_format can be set to any callable that

[...] accept a floating point number and return a string with the desired format of the number. This is used in some places like SeriesFormatter. See core.format.EngFormatter for an example.

Niko Föhr
  • 28,336
  • 10
  • 93
  • 96
1

In your second screenshot the second value in column B is 0.22 which is correctly rounded then to 0.2. All values in the second screenshot round to 0.x0. So the missing last digit is a feature from the GUI, suppressing a trailing 0.

The error is likely not in the rounding to 0.05. It is before that.

It appears as if the rounding to two digits using round(2) is not applied to the input in your example (the second value in B in your example is 0.15779.

Christian Fries
  • 16,175
  • 10
  • 56
  • 67
1

Pandas has this thing which removes trailing zeros for digits after trailing zeros. I guess its sort of a feature or a bug. If you just want to see the output to the right precision on your display/print, have you tried the display_precison option, like

pd.set_option('precision', 2)

Or change 2 to 3 or 4 to play around. I think this is globally display precision option though, so if you want to display different precision for different column, that will be a problem.

Jim
  • 36
  • 2