0

How Normalize Data Mining MinMax from csv in Python 3 with library this is example of my data

RT      NK    NB    SU    SK    P    TNI IK   IB     TARGET
84876   902  1192  2098  3623  169   39  133  1063   94095
79194   902  1050  2109  3606  153   39  133   806   87992
75836   902  1060  1905  3166  161   39  133   785   83987
75571   902   112  1878  3190  158   39  133   635   82618
83797  1156   134  1900  3518  218   39  133   709   91604
91648  1291   127  2225  3596  249   39  133   659   99967
79063  1346   107  1844  3428  247   39  133   591   86798
84357  1018   122  2152  3456  168   39  133   628   92073
90045   954   110  2044  3638  174   39  133   734   97871
83318   885   198  1872  3691  173   39  133   778   91087
93300  1044   181  2077  4014  216   39  133   635  101639
88370  1831   415  2074  4323  301   39  133   502   97988
91560  1955   377  2015  4153  349   39  223   686  101357
85746  1791   314  1931  3878  297   39  215   449   94660
93855  1891   344  2064  3947  287   39  162   869  103458
97403  1946   382  1937  4029  289   39  122  1164  107311

the formula MinMax is 

= (data-min)/(max-min)*0.8+0.1

i got the code but the normalize data is not each column

I know how to count it like this

(first data of RT - min column RT data) / (max column RT- min column RT) * 0.8 + 0.1, etc

so does the next column 

(first data of NK - min column NK data) / (max column NK- min column NK) * 0.8 + 0.1

like this please help me

this is my code, but i don't understand

from sklearn.preprocessing import Normalizer
from pandas import read_csv
from numpy import set_printoptions
import pandas as pd

#df1=pd.read_csv("dataset.csv")
#print(df1)

namaFile = 'dataset.csv'
nama = ['rt', 'niagak', 'niagab', 'sosum', 'soskhus', 'p', 'tni', 'ik', 'ib', 'TARGET']
dataFrame = read_csv(namaFile, names=nama)
array = dataFrame.values

#membagi array
X = array[:,0:10]
Y = array[:,9]

skala = Normalizer().fit(X)
normalisasiX = skala.transform(X)

#data hasil
print('Normalisasi Data')
set_printoptions(precision = 3)
print(normalisasiX[0:5,:])

the results of manual counting with code are very different

3 Answers3

0

By using MinMaxScaler from sklearn you can solve your problem.

from pandas import read_csv
from sklearn.preprocessing import MinMaxScaler

df = read_csv("your-csv-file")
data = df.values

scaler = MinMaxScaler()
scaler.fit(data)

scaled_data = scaler.transform(data)
pcko1
  • 833
  • 12
  • 22
  • i tried but like this "C:\Users\Dini\Anaconda3\lib\site-packages\sklearn\utils\validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by MinMaxScaler. warnings.warn(msg, DataConversionWarning)" –  Mar 08 '19 at 08:59
  • is this for each column? –  Mar 08 '19 at 09:03
  • add `df = df.astype(float)` after `read_csv()` to cast your dataframe to float – pcko1 Mar 08 '19 at 09:03
  • can you see my post https://stackoverflow.com/questions/55084336/how-normalize-data-mining-min-max-from-mysql-in-python in another my account? because this account was banned so I can't ask more questions @pcko1 –  Mar 10 '19 at 04:20
0

we can use pandas python library.

import pandas as pd

df = pd.read_csv("filename")

norm = (df - df.min()) / (df.max() - df.min() )*0.8 + 0.1

norm will have the normalised dataframe

newlearnershiv
  • 350
  • 1
  • 9
  • is this for each column? –  Mar 08 '19 at 09:01
  • norm will have every column in the csv file. values will be min-max normalized. we needn't apply it for individual columns. – newlearnershiv Mar 08 '19 at 09:03
  • thank you for help, the answer of code is same with my counting manual –  Mar 08 '19 at 09:13
  • can you see my post https://stackoverflow.com/questions/55084336/how-normalize-data-mining-min-max-from-mysql-in-python in another my account? because this account was banned so I can't ask more questions @newlearnershiv –  Mar 10 '19 at 04:19
0
import matplotlib.pyplot as plt   
import pandas as pd
from sklearn.cluster import KMeans  
from pandas import DataFrame
from sklearn.preprocessing import MinMaxScaler

data = pd.read_csv('Q4dataset.csv')
#print(data)
df = DataFrame(data,columns=['X','Y'])
scaler = MinMaxScaler()
scaler.fit(df)
#print(scaler.transform(df))
minmaxdf = scaler.transform(df)
kmeans = KMeans(n_clusters=2).fit(minmaxdf)
centroids = kmeans.cluster_centers_
plt.scatter(df['X'], df['Y'], c= kmeans.labels_.astype(float), s=30, alpha=1)

You can use the code I wrote above. I performed min-max normalization on two-dimensional data and then applied K means clustering algorithm.Be sure to include your own data set in .csv format

drorhun
  • 500
  • 7
  • 22