before I begin I've seen: Perceptron implementation, decision boundary will not plot
and:
Plot a 3D Boundary Decision in Python
and I still can't figure this out!
I've trained a perceptron to classify flower class based on the classic IRIS dataset. I want to iteratively plot a 3d decision boundary over successive epochs so that I can 'see' the model updating. Here is an example of something similar I've done for a polynomial regression: Updating the model parameters over epochs of gradient descent
I understand the mathematical intuition behind plotting a plane as a decision boundary. There are three input dimensions, so the clusters of each class could be separated by a plane. I understand that the equation for such a plane is iteratively updated over successive epochs of training, and it's this updating that I want to visualize!
I'm at the point where the model is trained, and I have a matrix of weights. Here is my code, which you should be able to run immediately as the data is pulled directly from a URL:
#The perceptron has many inputs(features)
#The inputs are fed into a linear unit and produce one binary output
#The perceptron finds linear separation of a dataset, and can classify via decision boundary
import numpy as np
import pandas as pd
pd.options.mode.chained_assignment = None # default='warn'
import matplotlib.pyplot as plt
def load_data():
#read in data
URL_='https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data'
raw_data = pd.read_csv(URL_, header = None)
data = raw_data.rename(columns={0:'sepal_length_cm',1:'sepal_width_cm',\
2:'petal_length_cm',4:'flower_class'})
#drop petal width so we can easily visualize
data = data.drop([3], axis=1)
#We will only use two of the flowers because they are linearly seperable
training_data = data.iloc[0:100]
#label the data according to flower identity (0) is true for Iris-setosa
training_data['flower_class'] = np.where(training_data\
.iloc[:, -1]=='Iris-setosa', 0, 1)
#split the dataframe into separate test/train sets
test_1 = training_data.iloc[0:15,:]
test_2 = training_data.iloc[50:65,:]
frames = [test_1,test_2]
testing_data = pd.concat(frames)
training_data.drop(testing_data.index, axis=0,inplace=True)
#return data as a matrix of floats
training_data = np.asmatrix(training_data, dtype = 'float64')
return training_data,testing_data
def visualize(data):
#visualize the data features
plt.figure("Input Features",figsize=(11,7))
ax = plt.axes(projection ='3d')
ax.zaxis.set_major_formatter('{x:.02f}')
ax.scatter(np.array(data[:34,0]), np.array(data[:34,1]),\
np.array(data[:34,2]),label='setosa')
ax.scatter(np.array(data[35:,0]), np.array(data[35:,1]), \
np.array(data[:35,2]),marker='x', label='versicolor')
plt.legend(loc="upper left")
plt.show()
ax.set_xlabel('sepal length (cm)', fontweight ='bold')
ax.set_ylabel('sepal width (cm)', fontweight ='bold')
ax.set_zlabel('petal length (cm)', fontweight ='bold')
plt.title("Input features of Iris Flowers")
def perceptron(data, num_iter):
features = data[:, :-1]
labels = data[:, -1]
learning_rate = .1
# set weights to zero
w = np.zeros(shape=(1, features.shape[1]+1)) #initialize weights/bias to 0
print('\nInitialized synaptic weights / activation bias: ',w,'\n')
for epoch in range(num_iter):
for x, label in zip(features, labels): #go through each x value
x = np.insert(x,0,1) #insert 1 as a 'bias-weight' for dot product
y = np.dot(w, x.transpose()) #prediction, here is the linear combination
p_output = 1.0 if (y > 0) else 0.0 #threshold function
error = (label - p_output)
if error!=0: # misclassified - don't update params if correct output
w += learning_rate * (error * x)
print('epoch',epoch+1,' - Updated parameters: ',w)
return (w)
def main():
global data
training_data,testing_data = load_data()
visualize(training_data)
num_iter = 10
global w
w = perceptron(training_data, num_iter)
#solve for plane equation and plot decision boundary plane iteratively
main()
And here is my output:
Initialized synaptic weights / activation bias: [[0. 0. 0. 0.]]
epoch 1 - Updated parameters: [[0.1 0.67 0.31 0.44]]
epoch 2 - Updated parameters: [[ 0. 0.23 -0.21 0.6 ]]
epoch 3 - Updated parameters: [[ 0. 0.33 -0.34 0.89]]
epoch 4 - Updated parameters: [[ 0. 0.43 -0.47 1.18]]
epoch 5 - Updated parameters: [[-0.1 -0.14 -0.91 1.03]]
epoch 6 - Updated parameters: [[-0.1 -0.14 -0.91 1.03]]
epoch 7 - Updated parameters: [[-0.1 -0.14 -0.91 1.03]]
epoch 8 - Updated parameters: [[-0.1 -0.14 -0.91 1.03]]
epoch 9 - Updated parameters: [[-0.1 -0.14 -0.91 1.03]]
epoch 10 - Updated parameters: [[-0.1 -0.14 -0.91 1.03]]
How would I go about plotting a plane decision-boundary given these weights I've solved for?
