I am coding a neural network using the swish activation function and had the idea of doing the backpropagation using recursion. Here is the code for the portion where I deal with the coefficients beta inside the swish functions:
import numpy as np
def beta_gradient(depth,weight_matrices,bias_vectors,beta_vectors,layers):
weight_matrix = weight_matrices[-1]
bias_vector = bias_vectors[-1]
beta_vector = beta_vectors[-1]
len_current_layer = len(beta_vector)
if depth == 0:
beta_gradient = np.zeros((len_current_layer,len_current_layer))
for i in range(len_current_layer):
beta_gradient[i,i] = swish_gradient(beta_vector[i],np.dot(weight_matrix[i,:],layers[-2])+bias_vector[i])[0]
return beta_gradient
else:
diagonal = np.zeros((len_current_layer,len_current_layer))
for i in range(len_current_layer):
diagonal[i,i] = swish_gradient(beta_vector[i],np.dot(weight_matrix[i,:],layers[-2])+bias_vector[i])[1]
beta_gradient = diagonal.dot(weight_matrix).dot(beta_gradient(depth-1,weight_matrices[:-1],bias_vectors[:-1],beta_vectors[:-1],layers[:-1]))
return beta_gradient
weights = [np.array([[1,1],[2,2],[3,3]]), np.array([[1,1,1],[2,2,2]])]
biases = [np.zeros(3), np.zeros(2)]
betas = [np.ones(3),np.ones(2)]
layers = [np.ones(2),np.ones(3),np.ones(2)]
beta_gradient(1,weights,biases,betas,layers)
When I run this code I get the following error
UnboundLocalError: local variable 'beta_gradient' referenced before assignment
The code is a bit involved. However, my question is simple: if I change the name of the variable beta_gradient (inside the function beta_gradient()) to something else (say I call it "gradient"), then the code runs ok. Would anyone know why this happens? Thanks in advance.
