3

I am using sklearn StandardScaler to transform/normalize data, as the following:

scaler = StandardScaler()
data = scaler.fit_transform(data)

I am expecting a mean of 0 and a standard deviation of 1. However, the values I get are bit different.

rnd = randrange(0, data.shape[1])
print(data[:,rnd].std())
print(data[:,rnd].mean())

1.0282903146389404
-0.06686584736835668

It seems very close numbers to 0 and 1 should be acceptable; however, not sure what is the acceptable offset. For instance, +/- 1e-2, as I'm getting, is close enough? or I should be concerned?

Dr. Strangelove
  • 2,725
  • 3
  • 34
  • 61

1 Answers1

2

You are using fit_transform for you data variable. This means, that now, all your data will normalized, following a shape of mean 0 and standard deviation of 1. Like the image below:

enter image description here

What you are doing next, is taking, randomly, some samples of your data variable. So, the new sample you collected, will be almost identical, but, since there's a randomly parameters, the mean and the standard deviation will not be the same as your data.

To make a comparison, imagine that we have the mean and std of the human height. If we now, take a small sample of your country height, the mean and std won't be exactly the same, but almost. That's the point.

For sure if your check mean and std for your data variable, you will obtain mean 0 and std 1.

Alex Serra Marrugat
  • 1,849
  • 1
  • 4
  • 14