0

I was reading the following tutorial for testing proportions in two populations. After running

prop.test(x=c(342,290), n=c(400,400))

I received a p-value of 9.558674e-06, which the tutorial says is greater than the alpha level of .05. I assumed this was a typo, and was just comparing the p-value to its value in decimal notation, 0.000009558674, but received "False". I even turned off scientific notation using

options(scipen=999)

and when printing out the p-value from the object returned by prop.test, I still receive "False" when comparing the p-value to 0.000009558674 for equality, it recognizes the p-value as lesser than. Why is this the case?

Jaigus
  • 1,422
  • 1
  • 16
  • 31
  • 1
    My understanding of your question is that you are comparing 9.559e-06 (rounded) to 9.558674e-06, which could be less than or greater than depending on the unrounded value? – njp Dec 20 '21 at 22:23
  • 1
    Floating-point *equality* (especially with high precision numbers) should be tested with care; a better comparison is to see if the absolute-difference is below a threshold. For instance, `abs(9.559e-06 - 0.000009558674) < 1e-9` is true, indicating that those numbers are very similar. For floating-point equality, see https://stackoverflow.com/q/9508518, https://stackoverflow.com/q/588004, and https://en.wikipedia.org/wiki/IEEE_754. – r2evans Dec 20 '21 at 22:31
  • Sorry, there was a typo in the original post. The p-value I received was 9.558674e-06. – Jaigus Dec 20 '21 at 22:34
  • See r2evans answer. Equality of floating point numbers can never really be tested properly. – njp Dec 20 '21 at 22:37
  • 4
    This is a frequently asked question. See the R FAQ: https://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-doesn_0027t-R-think-these-numbers-are-equal_003f – G. Grothendieck Dec 20 '21 at 22:45
  • Thanks for the clarification everyone. – Jaigus Dec 21 '21 at 02:42

1 Answers1

1

You may want to consider using the all.equal() function. The tolerance between the values can be set with the tolerance argument.

isTRUE(all.equal(2, 2.00000001))
##  [1] TRUE
isTRUE(all.equal(2, 2.00000001, tolerance = 0.0000000001))
##  [1] FALSE
J.P. Le Cavalier
  • 1,315
  • 7
  • 16